Retrieving personalized visual content items in real time for display on digital-content-display devices within a physical space

ABSTRACT

This disclosure describes embodiments of methods, systems, and non-transitory-computer readable media that personalize visual content for display on digital signage near a projected location of a person by mapping visual content to physical items selected by the person. In some examples, the disclosed system identifies physical items selected by a person based on signals from the physical items, such as signals emitted by RFID tags affixed to (or other devices associated with) the physical items. The disclosed system analyzes the collection of physical items—as identified by the signals—to tailor digital signage content specific to the person. The disclosed system further tracks the location of the person as the person moves through a physical space and interacts with the physical items. Based on the tracked positions, the disclosed system determines a digital sign in proximity to a predicted location of the person to display the personalized visual content.

BACKGROUND

In recent years, both hardware and software have improved the display ofimages, videos, or other visual content on digital signs. Such digitalsigns may include digital billboards, tablets, marquees, menu boards,kiosks, and more. For example, conventional digital signage systemsfrequently utilize digital signs to publish digital content for targetedgroups expected to be at a given location. Often, conventional systemsutilize digital signs and displays within various types of physicalspaces, such as stores, showrooms, and other public places. Conventionalsystems often present and update content items via the digital signs.

Notwithstanding these improvements in digital signage, conventionaldigital signage systems often cannot dynamically select content todisplay but follow a set schedule or rigid set of rules controlled by acomputing model. More specifically, conventional systems often displayvisual content that is irrelevant to passersby and lack a computingmodel that can depart from a rigid set of rules. While some conventionalsystems attempt to display digital content that is specific to a targetviewer, such systems often inaccurately categorize or label the targetedviewer or compromise digital security by capturing personal information.

For instance, some conventional digital signage systems utilize camerasto capture images of a passing person's face and infer attributes aboutthe passing person. But such conventional systems often cannot collectaccurate biometric data. To effectively collect biometric information,cameras must often be positioned to capture a person's face. Even whencarefully positioned, cameras often fail to collect usable data forpeople whose faces are obscured (e.g., by other people, objects,clothing), not captured at the correct angle (e.g., from a side view),or are not entirely in frame.

In addition to technical obstacles to capturing biometric data,conventional digital signage systems often can infer only a limited setof attributes from information captured by camera. For example,conventional systems often fail to infer attributes beyond a person'sgender and age. Thus, conventional systems often face difficulty intailoring displayed content for a person based on flawed biometric dataand at the cost of capturing personal information that must be encryptedor otherwise protected against digital security breaches.

Some conventional digital signage systems attempt to further personalizedisplayed digital content by accessing personal data from securecomputing sources. For instance, conventional systems often utilizepersonal data accessed via computing devices associated with a person.However, conventional systems that access user computing devices oftenface privacy and other data security issues. In particular, conventionalsystems often face concerns in privacy when accessing a user's profileto retrieve personal information, such as the person's interests, name,age, gender, and related data. Furthermore, conventional systems mustoften store accessed and recorded information that creates a digitalsecurity risk.

Furthermore, due in part to the reliance on limited data, conventionaldigital signage systems often rigidly operate on after-the-fact datacaptured beyond a time window for real-time use and use a generalizedcomputing model that selects content based on averages of suchafter-the-fact data. For example, some conventional systems collect datafor persons, for example, by monitoring items bought at the point ofsale. Because conventional systems determine general patterns based onaverages from collected data, they are frequently limited to inflexiblydisplaying the same content designed for general classes of people.Thus, conventional systems are often incapable of dynamically displayingpersonalized content in real time to individual persons.

BRIEF SUMMARY

This disclosure describes one or more embodiments of systems, methods,and non-transitory computer readable storage media that provide benefitsand/or solve one or more of the foregoing problems. In particular, thedisclosed systems personalize visual content for display on digitalsignage by mapping visual content to physical items selected by a personand displaying the visual content on a particular digital sign nearby aprojected location of the person. For instance, the disclosed systemidentifies and categorizes a person based on a collection of physicalitems selected by the person, such as clothing, toiletries, or otherphysical products. In some examples, the disclosed system identifiesphysical items in the collection based on signals from radio-frequencyidentification (RFID) tags affixed to the physical items or signals fromother reflectors or devices associated with the physical items. Thedisclosed system analyzes the collection of physical items to tailordigital-signage content specific to the people. For example, thedisclosed system can infer that a person has a particular age, bodytype, gender, purpose, or spend level based on the collection of items.In addition to analyzing the selected physical items, the disclosedsystem tracks the location of the people as they move through a space(e.g., a store) and interact with the or otherwise select physicalitems. Based on the tracked positions, the disclosed system determines adigital sign in proximity to a predicted location of the people at apredicted time. The disclosed system can accordingly present, at adigital sign in the people's predicted location, personalizeddigital-signage content in real time.

Additional features and advantages of one or more embodiments of thepresent disclosure will be set forth in the description which follows,and in part will be obvious from the description, or may be learned bythe practice of such example embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments will be described and explained with additionalspecificity and detail through the use of the accompanying drawingswhich are summarized below.

FIG. 1 illustrates an environment in which apersonalized-visual-content-display system can operate in accordancewith one or more embodiments of the present disclosure.

FIG. 2 illustrates an overview of thepersonalized-visual-content-display system personalizing visual contentfor display on a digital-content-display device by mapping visualcontent to physical items selected by a person and displaying the visualcontent on a selected digital-content-display device nearby a projectedlocation of the person in accordance with one or more embodiments of thepresent disclosure.

FIG. 3 illustrates an example physical space in which a person interactswith or otherwise selects physical items and in which thepersonalized-visual-content-display system displays personalized visualcontent to the person on a digital-content-display device in accordancewith one or more embodiments of the present disclosure.

FIG. 4 illustrates an overview of thepersonalized-visual-content-display system determining a projectedlocation at which a person will arrive at a projected time in accordancewith one or more embodiments of the present disclosure.

FIG. 5A illustrates example sensors utilized by thepersonalized-visual-content-display system to determine a set ofphysical items selected by a person in accordance with one or moreembodiments of the present disclosure.

FIG. 5B illustrates the personalized-visual-content-display systemdetermining motion events and movement of physical items between areasin accordance with one or more embodiments of the present disclosure.

FIG. 6 illustrates an overview of thepersonalized-visual-content-display system retrieving a visual contentitem based on item embeddings and content embeddings in accordance withone or more embodiments of the present disclosure.

FIG. 7 illustrates an overview of thepersonalized-visual-content-display system retrieving a visual contentitem based on characteristics of the person in accordance with one ormore embodiments of the present disclosure.

FIG. 8 illustrates an overview of thepersonalized-visual-content-display system retrieving a visual contentitem using a co-selected-nodal graph in accordance with one or moreembodiments of the present disclosure.

FIG. 9 illustrates an overview of thepersonalized-visual-content-display system modifying a digital contentcampaign based on analyzing behaviors of the person in accordance withone or more embodiments of the present disclosure.

FIG. 10 illustrates a schematic diagram of an example architecture ofthe personalized-visual-content-display system in accordance with one ormore embodiments of the present disclosure.

FIG. 11 illustrates a series of acts for personalizing visual contentfor display on a digital-content-display device by mapping visualcontent to physical items selected by a person and displaying the visualcontent on a selected digital-content-display device nearby a projectedlocation of the person in accordance with one or more embodiments of thepresent disclosure.

FIG. 12 illustrates a block diagram of an example computing device inaccordance with one or more embodiments of the present disclosure.

DETAILED DESCRIPTION

This disclosure describes one or more embodiments of apersonalized-visual-content-display system that (i) retrieves apersonalized visual content item for display via adigital-content-display device in real time based on physical itemsselected by a person and (ii) selects a digital-content-display devicenearby a projected location of the person to display the personalizedvisual content item. In particular, thepersonalized-visual-content-display system tracks physical items that aperson interacts with, carries, or otherwise selects as the person movesthrough a physical space. In one example, thepersonalized-visual-content-display system receives signals from tagsaffixed to (or other devices associated with) individual physical items.The personalized-visual-content-display system analyzes physical itemsselected by the person to infer characteristics of the person and/orpredict other items in which the person has interest. Thepersonalized-visual-content-display system selects a visual content itemspecific to the person based on analysis of the selected physical items.As the person travels through the physical space, thepersonalized-visual-content-display system tracks the location of theselected physical items (or the person) and projects a path of theperson. Having projected a path and selected a visual content item, thepersonalized-visual-content-display system identifies adigital-content-display device in the projected path of the person todisplay the selected visual content item.

To illustrate, in one or more embodiments, thepersonalized-visual-content-display system determines a set of physicalitems selected by a person within a physical space based on signals fromthe set of physical items. The personalized-visual-content-displaysystem also identifies a description of one or more physical items fromthe set of physical items selected by the person. Thepersonalized-visual-content-display system further determines aprojected location of the person within the physical space at aprojected time and identifies a digital-content-display device inproximity to the projected location. Based on determining a visualcontent item corresponds to the description of one or more physicalitems, the personalized-visual-content-display system retrieves, fordisplay via the digital-content-display device, a visual content item tobe viewed by the person at the projected location and at the projectedtime.

As part of displaying personalized content items via adigital-content-display device, the personalized-visual-content-displaysystem determines identities of physical items within a set of physicalitems selected by a person. For instance, in some embodiments, a personcan place a physical item in a shopping container, such as a basket orcart, move or manipulate a physical item, or otherwise select a physicalitem to be part of the set of physical items. In one example, thepersonalized-visual-content-display system determines that individualitems traveling together as a group belong to the set of physical items.

As or after the person selects physical items, thepersonalized-visual-content-display system determines identifiers forindividual physical items within the set of physical items. To determinesuch identifiers, in one example, thepersonalized-visual-content-display system receives signals from thephysical items (e.g., RFID signals from tags affixed to the physicalitems). Additionally, or alternatively, thepersonalized-visual-content-display system utilizes computer vision,smart shelving, Quick Response (QR) codes, or other methods to identifyphysical items. The personalized-visual-content-display system canutilize various sensors and methods (or a combination thereof) to detectsignals from the physical items to identify the individual physicalitems. For instance, in some cases, thepersonalized-visual-content-display system utilizes sensors located onan item-holding structure, such as a container or shelf. In anotherexample, the personalized-visual-content-display system utilizes sensorscomprising RFID readers or other types of wireless communication signalreaders.

As mentioned, the personalized-visual-content-display system retrieves avisual content item that is tailored to the person based on the set ofphysical items selected by the person. In one example, thepersonalized-visual-content-display system displays a visual contentitem based on similarities between the visual content item and the setof physical items. In some cases, thepersonalized-visual-content-display system extracts a set of itemfeatures from a set of descriptions corresponding to the set of physicalitems. The personalized-visual-content-display system also extracts aset of content features from a collection of visual content items. Basedon mapping the set of content features to the set of item features, thepersonalized-visual-content-display system retrieves a visual contentitem to display to a person on a digital-content-display device. Forexample, in at least one embodiment, thepersonalized-visual-content-display system extracts a set of itemfeatures by generating item embeddings utilizing a language-embeddingmodel and extracts a set of content features by generating a contentembedding utilizing an image-embedding model. Thepersonalized-visual-content-display system maps the set of contentfeatures to the set of item features by determining measures ofsimilarities between the content embedding and the item embeddings.

In addition or in the alternative to mapping features to select a visualcontent item, in some embodiments, thepersonalized-visual-content-display system utilizes a co-selected-nodalgraph to identify a visual content item that is customized to theperson. For instance, the personalized-visual-content-display systemaccesses co-selected-nodal graphs that indicate probabilities thatparticular physical items are co-selected (e.g., co-viewed and/orco-bought) with the one or more physical items in the set of physicalitems. Based on historical data from the co-selected-nodal graphs, thepersonalized-visual-content-display system determines the likelihoodthat the person will view or select a commonly selected physical item.After selecting such an item, the personalized-visual-content-displaysystem determines to display a visual content item showing the commonlyselected physical item.

In addition or in the alternative to using a co-selected-nodal graph toselect a visual content item, in one or more embodiments, thepersonalized-visual-content-display system retrieves personalized visualcontent items based on inferred characteristics of the person. In atleast one example, the personalized-visual-content-display systemanalyzes the set of physical items and determines characteristics of aperson. For instance, the personalized-visual-content-display systeminfers the person's age, gender, spend level, size, shopping purpose, orother characteristics based on the set of physical items. Thepersonalized-visual-content-display system further identifies andretrieves a visual content item that match characteristics of theperson. For example, the personalized-visual-content-display systemselects a visual content item that shows a human model similar in age,gender, and/or size to the person.

Beyond tracking and analyzing physical items selected by a person, asindicated above, the personalized-visual-content-display system trackslocations of the selected physical items or the person. For instance,the personalized-visual-content-display system also tracks locations ofthe set of physical items as it travels within a physical space. In someembodiments, the personalized-visual-content-display system analyzessignals received from the set of physical items to determine thelocations. In one example, the personalized-visual-content-displaysystem identifies locations corresponding to electromagnetic signals(e.g., RFID signals) from devices on the set of physical items to trackdetected locations of the set of physical items. Based on the detectedlocations, the personalized-visual-content-display system generates aprojected path of the person by identifying historical paths of pastpersons with similar trajectories. Generally, the projected pathindicates projected locations that the person will likely go within thespace at projected times.

In some embodiments, the personalized-visual-content-display systemutilizes the projected path to identify a digital content-display devicein the projected path of the person. Based on the projected path, thepersonalized-visual-content-display system determines a projectedlocation at which the person will arrive at a projected time. Thepersonalized-visual-content-display system utilizes the projected pathto identify a digital-content-display device in proximity to a projectedlocation of the person (e.g., closest to the projected location orwithin a threshold distance). Together with a projected time for theprojected location, the personalized-visual-content-display systempredicts digital-content-display devices that the person will likely seenext or soon within the physical space.

As previously mentioned, the personalized-visual-content-display systemprovides numerous advantages, benefits, and practical applications overconventional digital signage systems. In particular, thepersonalized-visual-content-display system more accurately tailorsvisual content items to individual persons relative to conventionalsystems as the person travels through a physical space in real time on anearby digital-content-display device. As mentioned previously,conventional systems are often limited to presenting customized digitalcontent at a point of sale or at some other location for a generalizedpopulation. In contrast, the personalized-visual-content-display systemaccurately projects a future location and a future time at which theperson will view a given digital-content-display device. Morespecifically, the personalized-visual-content-display system analyzesitems within the set of physical items in real time to select visualcontent items that are specifically customized to the person at anygiven moment as the person travels through a physical space. Forexample, the personalized-visual-content-display system displays visualcontent items that are specific to the person's shopping trip, size,age, gender, and other characteristics and customized for a specificprojected location that the person will view at a particular projectedtime.

In addition to improved accuracy of tailored content in real time andphysical space, in some embodiments, thepersonalized-visual-content-display system performs an orderedcombination of unconventional actions as part of an algorithm to dosomething conventional digital signage systems cannot—display customizedvisual content items to a person in real time as the person movesthrough a physical space. In particular, thepersonalized-visual-content-display system integrates RFID signals (orother signals associated with physical items) to determine theidentities of physical items selected by a person and combined to form aset of physical items. As part of the ordered combination of actions, insome cases, the personalized-visual-content-display system identifies avisual content item corresponding to the set of physical items bymapping a set of content features from the visual content item to a setof item features from the set of physical items or by utilizing aco-selected-nodal graph to find co-selected items in the visual contentitem. Rather than rely on human targeting or conventionaldigital-signage campaigns, therefore, thepersonalized-visual-content-display system performs an ordered set ofunconventional actions to display customized visual content items inreal time to a person—based on signals and extractions unrecognizable byhumans—as the person moves through a physical space.

Beyond personalized accuracy and an unconventional algorithm toaccomplish such personalization, the personalized-visual-content-displaysystem further provides such accuracy without compromising privacy ordata of a person. More specifically, in contrast with conventionalsystems that capture images of a person's face, thepersonalized-visual-content-display system collects data regardingphysical items that a person has selected within a physical space. Thepersonalized-visual-content-display system does so by receiving andanalyzing signals associated with the physical items. For instance, thepersonalized-visual-content-display system receives signals from RFIDtags affixed to physical items, smart shelves, or other types of sensorslocated within the physical space. By analyzing the set of physicalitems, the personalized-visual-content-display system entirelycircumvents the capture and storage of biometric data typically used byconventional systems. Accordingly, thepersonalized-visual-content-display system avoids encryption, storage,and other security risks often faced by conventional systems.

Independent of improved data security, thepersonalized-visual-content-display system improves the flexibility withwhich systems can personalize a visual content item shown on adigital-content-display device in real time for a particular personmoving through a physical space. In particular, thepersonalized-visual-content-display system makes dynamic determinationsthat are flexible for various situations. In particular, thepersonalized-visual-content-display system functions flexibly byadapting visual content items to individual persons based on physicalitems selected by a person as the person moves or interacts with aphysical space. For example, the personalized-visual-content-displaysystem dynamically updates projected locations and customized visualcontent items in real time as the person travels through the physicalspace. Furthermore, in some embodiments, thepersonalized-visual-content-display system integrates signals from thephysical items to both adjust the selected visual content item as wellas determine the projected path of the person. For example, based ondetermining that a person adds a physical item to the set of items, thepersonalized-visual-content-display system retrieves (or updates) avisual content item related to the selected item and adjusts theprojected path. Unlike conventional systems that are often limited tocollecting user data at a final point of sale, thepersonalized-visual-content-display system offers visual content itemsrelevant to a person's current visit to the physical space.

Additional advantages and benefits of the artistic effect generationsystem will become apparent in view of the following description.Further, as illustrated by the foregoing discussion, the presentdisclosure utilizes a variety of terms to describe features andadvantages of the artistic effect generation system. The followingparagraphs provide additional detail regarding the meaning of suchterms.

As used herein, the term “physical item” refers to a physical objectthat exists in the real world. In particular, a physical item includes aphysical and tangible object within a physical space. For example, aphysical item includes physical products within a store, a physical goodin an inventory, or another type of tangible object within a physicalspace. Such physical items may be, but are not limited to, physicalaccessories, books, clothes, electronic devices, toiletries, or anyother tangible or physical object in the real world. In someembodiments, physical items are associated with wireless protocol tags.For example, a physical item can be attached to or otherwise associatedwith an RFID tag, a visual QR code, an ultra-wideband (UWB) transmitter,or other type of wireless protocol tag. In additional or alternativeembodiments, physical items are not associated with tags, such as when acamera or computer vision identifies physical items.

As used herein, the term “item feature” refers to a characteristic orattribute of an item. In particular, item features comprise valuescorresponding to latent and/or patent attributes and characteristics ofa physical item within a physical space. In some embodiments, itemfeatures comprise words or concepts extracted from a description of aphysical item. Additionally, or alternatively, item features comprisevectors (e.g., a series of numbers) extracted from a description of aphysical item for a word embedding.

As used herein, the term “description” refers to a textualrepresentation of a physical item. In particular, a description includesa written description of a type of physical item or characteristics orattributes of a physical item. A description for a physical itemindicates a sort, kind, or class of a physical item. For example, adescription might indicate that a physical item is an article of women'sclothing, medicine, an electronic device, or other type of item.Furthermore, in some embodiments, the description provides additionaldetail regarding the corresponding physical item. For instance, thedescription can indicate the size, color, style, type, or otherattributes of the physical item.

As used herein, the term “physical space” refers to an area that existsin the physical or real world. In particular, a physical space comprisesan indoor or outdoor physical demarcated by boundaries that containphysical items. For example, a physical space can comprise a store thatincludes goods for sale, such as a supermarket, shopping center, drugstore, etc. In other embodiments, a physical space includes a structurefor storing and organizing inventory such as a warehouse.

As used herein, the term “signal” refers to a current, frequency, pulse,or wave used to carry data from one device to another device. Inparticular, a signal includes a current, frequency, pulse, or wave thatrepresents data associated with a physical item. In one or moreembodiments, signals are emitted or reflected by tags affixed tophysical items. In such examples, signals comprise radio signals emittedby RFID and/or UWB tags affixed to physical items. Additionally, oralternatively, signals are emitted by other types of sensors thatcommunicate the status of physical items. For example, signals may beemitted by other types of sensors associated with item-holdingstructures (e.g., shelves, racks) that indicate that a person hasremoved a physical item from the item-holding structures.

As used herein, the term “projected location” refers to a predictedlocation of a person or an object. In particular, a projected locationcomprises a place along a projected path or trajectory at which a personis predicted to be (or near) at a projected time. In some embodiments,the personalized-visual-content-display system generates a number ofprojected locations, each associated with a different projected time andin accordance with a projected path of the person.

As used herein, the term “digital-content-display device” (or simply“display device”) refers to a computing device that displays digitalcontent. In particular, a digital-content-display device can include acomputing device having a digital display for displaying and/or playingvisual content items. In some implementations, a digital-content-displaydevice houses and implements the personalized-visual-content-displaysystem. In alternative embodiments, the digital-content-display deviceis located remotely from the personalized-visual-content-display systemand displays visual content items upon or after receiving them from thepersonalized-visual-content-display system. In some embodiments, adigital-content-display device displays a visual content item uponreceiving a display command from the personalized-visual-content-displaysystem or, alternatively, upon or after receiving the visual contentitem.

As used herein, the term “visual content item” refers to digital mediacomprising an image or video. In particular, a visual content item caninclude digital media stored in a digital file that is transferablebetween computing devices as well as displayable at adigital-content-display device. For example, a visual content item caninclude still digital images, digital videos, gifs, or other visualdigital media. In some embodiments, visual content items are associatedwith metadata that reflect attributes and/or content of the visualcontent items. For example, metadata can describe the content of animage, the date and time when a photograph was taken, and otherinformation.

As used herein, the term a “content feature” refers to a characteristicor attribute of a visual content item. In particular, a content featurescomprise values corresponding to latent and/or patent attributes andcharacteristics of a visual content item. In some embodiments, contentfeatures comprise words or concepts that describe a visual content item.Additionally, or alternatively, in some cases, content features comprisevectors (e.g., a series of numbers) extracted from a visual content itemfor an image embedding.

The following disclosure provides additional detail regarding thepersonalized-visual-content-display system in relation to illustrativefigures portraying example embodiments and implementations of thepersonalized-visual-content-display system. For example, FIG. 1illustrates a schematic diagram of a system environment (or“environment”) 100 in which a personalized-visual-content-display system106 operates in accordance with one or more embodiments. As illustratedin FIG. 1, the environment 100 includes an administrator client device122, a server device(s) 102, digital-content-display devices 110 a-110n, and sensors 118 a-118 n connected via a network 112. In particular,the digital-content-display devices 110 a-110 n, sensors 118 a-118 n,and physical items 120-120 n are located within a physical space 108.

While FIG. 1 shows an embodiment of thepersonalized-visual-content-display system 106, alternative embodimentsand configurations are possible. For example, the environment 100 caninclude any number of client devices, servers, or other components incommunication with the personalized-visual-content-display system 106via the network 112. As another example, the server device(s) 102represent a set of connected server devices. As a further example, thedigital-content-display devices 110 a-110 n and the sensors 118 a-118 ncommunicate directly with the server device(s) 102, bypassing thenetwork 112 or utilizing a separate and/or additional network.

In some embodiments, the server device(s) 102, the network 112, thedigital-content-display devices 110 a-110 n, and the sensors 118 a-118 nare communicatively coupled with each other either directly orindirectly. For example, and as shown in FIG. 1, the server device(s)102 communicates with the various components of the environment 100 viathe network 112. Each of the components of the environment 100communicate via the network 112. The network 112 comprises any suitablenetwork over which computing devices can communicate. Example networksare discussed in additional detail below in relation to FIG. 12.

As illustrated in FIG. 1, the environment 100 includes the serverdevice(s) 102. The server device(s) 102 generate, store, and transmitdigital content including software hosted at the server device(s) 102.The server device(s) 102 generate, store, and transmit digital contentrelating to visual content items including digital video, digitalimages, digital audio, digital designs, metadata, etc. Furthermore, theserver device(s) 102 also receive, store, and transmit digital contentrelating to physical items within a physical space. In particular, theserver device(s) generate, store, and transmit data from the sensors 118a-118 n, identities of, and metadata associated with physical itemswithin the physical space 108. Furthermore, in some embodiments, theserver device(s) 102 generate, store, receive, and transmit data forneural networks.

As further shown in FIG. 1, the server device(s) 102 include a campaignmanagement system 104. Generally, the campaign management system 104manages, operates, runs, and/or executes one or more digital contentcampaigns. In one example, the campaign management system 104 receives,from the administrator client device 122, a collection of visual contentitems from which the personalized-visual-content-display system 106selects a visual content item to display via the digital-content-displaydevices 110 a-110 n. In some embodiments, the campaign management system104 provides, for display, particular visual content items as part ofspecific digital content campaigns. Furthermore, the campaign managementsystem 104 manages digital content campaigns specific to a particularperson or type of person. For example, the campaign management systemcan select specific visual content items to display to a specific personbased on data associated with the person. In one embodiment, thecampaign management system 104 generates digital campaigns based on aset of physical items associated with a person.

In some cases, the campaign management system 104 operates in connectionwith one or more applications to generate and modify digital contentcampaigns. For example, in one or more embodiments, the campaignmanagement system 104 operates in connection with digital contentcampaign applications, such as ADOBE® EXPERIENCE MANAGER, ADOBE®ADVERTISING CLOUD, ADOBE® ANALYTICS, ADOBE® MARKETING CLOUD, ADOBE®EXPERIENCE CLOUD, and other campaign management applications.

As further illustrated in FIG. 1, the campaign management system 104includes the personalized-visual-content-display system 106. Asdescribed in additional detail below, thepersonalized-visual-content-display system 106 retrieves personalizedvisual digital content items for display to a person based on a set ofphysical items associated with the person. In some embodiments, thepersonalized-visual-content-display system 106 determines a set ofphysical items selected by a person within a physical space based onsignals from the set of physical items. Thepersonalized-visual-content-display system 106 further identifiesdescriptions of physical items within the set of physical items.Furthermore, the personalized-visual-content-display system 106 predictsa projected location of the person at a projected time and identifies adigital-content-display device in proximity to the projected location ofthe person. The personalized-visual-content-display system 106 furtherretrieves visual content items to be displayed at thedigital-content-display device at the projected time based on the visualcontent item corresponding to the description of one or more physicalitems from the set of physical items.

As further illustrated in FIG. 1, the environment 100 includes theadministrator client device 122. The administrator client device 122generates, stores, receives, and sends digital data. For example, theadministrator client device 122 communicates with the server device(s)102 via the network 112. The administrator client device 122 illustratedin FIG. 1 comprises various types of client devices. For example, insome embodiments, the administrator client device 122 comprises mobiledevices, such as laptops, tablets, mobile telephones, smartphones, etc.In other embodiments, the administrator client device 122 includesnon-mobile devices, such as desktops or servers, or other types ofclient devices. In some embodiments, the administrator client device 122communicates with the campaign management system 104 via the network 112to coordinate and execute digital content campaigns. For example, theadministrator client device 122 may send, to the server device(s) 102,visual content items that may be displayed via thedigital-content-display devices 110 a-110 n. Additional details withregard to the administrator client device 122 are discussed below withrespect to FIG. 12.

The administrator client device 122 illustrated in FIG. 1 includes anapplication 114. In some embodiments, the application 114 comprises aweb application or a native application on the administrator clientdevice 122 (e.g., a mobile application, a desktop application). Asillustrated, the application 114 interfaces with thepersonalized-visual-content-display system 106 to provide digital datarelating to digital content campaigns. For example, in one or moreembodiments, the application 114 comprises a desktop application thatprovides features and elements for generating, modifying, and executingdigital content campaigns. More specifically, the application 114enables an administrator associated with the administrator client device122 to provide a set of visual content items from which thepersonalized-visual-content-display system 106 selects visual contentitems.

As further illustrated in FIG. 1, the environment 100 includes thephysical space 108. As mentioned previously, the physical space 108includes the digital-content-display devices 110 a-110 n, the sensors118 a-118 n, and the physical items 120 a-120 n. In certainimplementations, the digital-content-display devices 110 a-110 ncomprise a network of display devices located within the physical space108. In one example, the digital-content-display devices 110 a-110 ncomprise digital billboards located within a store. Thedigital-content-display devices 110 a-110 n receive visual content itemsfrom the personalized-visual-content-display system 106 via the network112 or receive visual content items from a local storage device thatinterfaces with the digital-content-display device 110 a-110 n.Furthermore, the digital-content-display devices 110 a-110 n store,manage, and display visual content items. For example, in oneembodiment, the digital-content-display devices 110 a-110 n receive anumber of visual content items for display to different persons. Thedigital-content-display devices 110 a-110 n update their user interfacesto present visual content items and update the user interfaces topresent additional visual content items at projected times.

The environment 100 also includes the sensors 118 a-118 n. Asillustrated in FIG. 1, the sensors 118 a-118 n are associated with thephysical items 120 a-120 n, respectively, but can alternatively beunassociated with any particular physical item. The sensors 118 a-118 ncollect, store, and communicate data relating to the physical items 120a-120 n. In particular, the sensors 118 a-118 n communicate with thepersonalized-visual-content-display system 106 via the network 112. Inone example, the sensors 118 a-118 n comprise radio sensorsstrategically located throughout the physical space 108. The radiosensors can receive signals (e.g., RFID, UWB) transmitted by tagsaffixed to the physical items 120 a-120 n. In another embodiment, thesensors 118 a-118 n comprise pressure sensors, cameras, or other typesof sensors within the physical space 108 that determine identities andlocations of the physical items 120 a-120 n. As mentioned, the sensors118 a-118 n communicate, to the personalized-visual-content-displaysystem 106, detected movement of the physical items 120 a-120 n withinthe physical space 108.

The environment 100 illustrated in FIG. 1 also includes the physicalitems 120 a-120 n. As mentioned, the physical items 120 a-120 n compriseobjects within the physical space 108. In one embodiment, the physicalitems 120 a-120 n comprise a set of physical items. More particularly,in one example, the physical items 120 a-120 n comprise merchandise orproducts that a person has selected. The person may move with thephysical items 120 a-120 n within the physical space 108 or manipulateor otherwise move the physical items 120 a-120 n.

Although FIG. 1 depicts the personalized-visual-content-display system106 located on the server device(s) 102, in some embodiments, thepersonalized-visual-content-display system 106 is implemented by (e.g.,located entirely or in part) on one or more other components of theenvironment 100. In one example, the personalized-visual-content-displaysystem 106 is implemented entirely (or in part) on the administratorclient device 122. In one or more embodiments, thepersonalized-visual-content-display system 106 is implemented in avariety of different ways across the server device(s) 102, the network112, the administrator client device 122, and thedigital-content-display devices 110 a-110 n.

In some embodiments, the personalized-visual-content-display system 106employs machine learning, such as a neural network, to infer what aperson who selects a set of physical items is most likely to select orpurchase in the future. In particular, in some embodiments, thepersonalized-visual-content-display system 106 learns parameters for avisual content item retrieval model based on training data comprisingitems selected and/or purchased by people. Thepersonalized-visual-content-display system 106 utilizes the visualcontent item retrieval model to generate predicted physical items basedon physical items within a set of physical items.

Machine learning refers to the process of constructing and implementingalgorithms that can learn from and make predictions on data. In general,machine learning may operate by building models from example inputs,such as image exposure training pairs within a training dataset ofimages, to make data-driven predictions or decisions. Machine learningcan include neural networks (e.g., a natural language processing neuralnetwork, a specialized object detection neural network, a concept-basedobject detection neural network, a known object class detection neuralnetwork, an object proposal neural network, an unknown object classdetection neural network, a region proposal neural network, a conceptembedding neural network, an object mask neural network, an objectclassification neural network, and/or a selected object attributedetection neural network), data-based models (e.g., a natural languageprocessing model, an unknown object class detection model, an objectrecognition model, a filtering model, and/or a selection objectattribute model), or a combination of networks and models.

FIG. 1 illustrates an example environment in which thepersonalized-visual-content-display system 106 in accordance with one ormore embodiments. In accordance with one or more embodiments, FIG. 2 andthe corresponding discussion provide a general overview of thepersonalized-visual-content-display system 106 personalizing visualcontent for display on a digital-content-display device by mappingvisual content to physical items selected by a person and displaying thevisual content on a selected digital-content-display device nearby aprojected location of the person. More specifically, FIG. 2 illustratesa series of acts 200 comprising an act 202 of determining a set ofphysical items, an act 204 of identifying a description of the physicalitems, an act 206 of determining a projected location for the person, anact 208 of identifying a digital content display device, and an act 210of retrieving a visual content item for display on thedigital-content-display device.

As illustrated in FIG. 2, the personalized-visual-content-display system106 performs the act 202 of determining a set of physical items. Forinstance, the personalized-visual-content-display system 106 determinesa set of physical items moved, manipulated, or otherwise selected by aperson based on signals received from the set of physical items. In someembodiments, the personalized-visual-content-display system 106 receivessignals indicating one or both of identifiers for the set of physicalitems and locations of physical items within the set of physical items212. In some cases, the personalized-visual-content-display system 106determines that the physical items A, B, and C have been moved (e.g.,into a container or area) or are co-located and thus belong to a set ofphysical items 212. For example, and as illustrated in FIG. 2, thepersonalized-visual-content-display system 106 determines that theperson 214 has picked up the physical items A, B, and C, and the person214 is moving through the physical space (e.g., a store) with thephysical items.

In one example, as part of performing the act 202, thepersonalized-visual-content-display system 106 determines that thephysical items A, B, and C belong to the set of physical items 212because the physical items A, B, and C are moving as a set within thephysical space. In one embodiment, thepersonalized-visual-content-display system 106 retrieves, fromelectromagnetic sensors at various locations within the physical space,indications of electromagnetic signals emitted by electromagnetic tagsassociated with the set of physical items 212. Thepersonalized-visual-content-display system 106 determines an identifierfor each physical item within the set of physical items 212. FIG. 5 andthe corresponding discussion provides additional detail regarding howthe personalized-visual-content-display system 106 receives signalsassociated with the set of physical items 212 in accordance with one ormore embodiments.

As further illustrated in FIG. 2, thepersonalized-visual-content-display system 106 performs the act 204 ofidentifying a description of the physical items. Generally, thepersonalized-visual-content-display system 106 determines identifiersfor the set of physical items 212. In one example, thepersonalized-visual-content-display system 106 determines identifiersfor the set of physical items 212 prior to purchase and/or before theperson 214 arrives at a point of sale. In some embodiments, thepersonalized-visual-content-display system 106 accesses or determinesdescriptions for one or more physical items from the set of physicalitems 212. For example, and as illustrated in FIG. 2, thepersonalized-visual-content-display system 106 determines that thephysical item A is associated with the description comprising “coldmedicine, adult.” The personalized-visual-content-display system 106 cando likewise for each selected physical item.

As indicated above, in one or more embodiments, thepersonalized-visual-content-display system 106 extracts item featuresfrom the descriptions of the one or more physical items from the set ofphysical items 212—as a basis for later retrieving a visual contentitem. FIG. 6 and the corresponding discussion provide additional detailregarding how the personalized-visual-content-display system 106extracts a set of item features from the descriptions of the physicalitems in accordance with one or more embodiments. In furtherembodiments, the personalized-visual-content-display system 106 utilizesthe descriptions of the physical items to determine characteristics ofthe person—or accesses a co-selected-nodal graph to determine a commonlyselected physical item—as a basis for later retrieving a visual contentitem.

As further illustrated in FIG. 2, thepersonalized-visual-content-display system 106 performs the act 206 ofdetermining a projected location 216 for the person 214. Generally, thepersonalized-visual-content-display system 106 analyzes the pasttrajectory of the person 214 to predict the future trajectory of theperson 214 within the physical space. In some embodiments, thepersonalized-visual-content-display system 106 generates a projectedpath of the person 214 within the physical space based on detectedlocations corresponding to the set of physical items 212. Thepersonalized-visual-content-display system 106 can utilize the samesignals analyzed as part of the act 202 in determining the locationscorresponding to the set of physical items 212.

To determine past locations of a person and a projected location, in oneor more embodiments, the personalized-visual-content-display system 106tracks the location of RFID tags of physical items in the set ofphysical items 212 as the person 214 travels through the physical space.Additionally, or alternatively, the personalized-visual-content-displaysystem 106 utilizes methods differing from analyzing signals from thephysical item to determine past locations of the person and theprojected location. For example, in some embodiments, thepersonalized-visual-content-display system 106 utilizes device locationdata indicating the location of a computing device associated with theperson, visual data from one or more cameras within the physical space,wireless (e.g., UWB) signals emitted by a container associated with theperson, and other sensor data to determine the locations of the set ofphysical items. FIGS. 3-4 and the corresponding discussion provideadditional detail with respect to predicting a projected location basedon a projected path of the person 214 in accordance with one or moreembodiments.

After determining the projected location, thepersonalized-visual-content-display system 106 performs the act 208 ofidentifying a digital-content-display device. Generally, thepersonalized-visual-content-display system 106 utilizes the projectedlocation of the person to identify a digital-content-display device thatthe person 214 is likely to view in the future. In particular, thepersonalized-visual-content-display system 106 identifies adigital-content-display device 218 in proximity to the projectedlocation 216 of the person 214 within the physical space. For example,the personalized-visual-content-display system 106 predicts that theperson 214 will travel to a particular intersection within the physicalspace and identifies the digital-content-display device at or near theparticular intersection.

As further illustrated in FIG. 2, thepersonalized-visual-content-display system 106 performs the act 210 ofretrieving a visual content item. Generally, as part of the act 210, thepersonalized-visual-content-display system 106 determines a visualcontent item to display via the digital-content-display device 218 thatmatches the person 214 based on the set of physical items 212. Inparticular, the personalized-visual-content-display system 106 selects,from a collection of visual content items, a visual content item todisplay via the digital-content-display device 218. More specifically,the personalized-visual-content-display system 106 retrieves the visualcontent item to be viewed by the person 214 at the projected locationand at the projected time. For example, as illustrated in FIG. 2, thepersonalized-visual-content-display system 106 provides, for display viathe digital-content-display device 218, a visual content item includingan advertisement for women's slacks. More specifically, thepersonalized-visual-content-display system 106 displays women's slacksthat are size 4, which matches the sizes of the physical items B and C.

In one or more embodiments, the personalized-visual-content-displaysystem 106 utilizes the set of item features extracted from adescription(s) of the set of physical items 212 to select the visualcontent item from the collection of visual content items. In particular,the personalized-visual-content-display system 106 infers that theperson 214 is likely interested in additional physical items that aresimilar to or otherwise related to the set of physical items 212. Forexample, in one or more embodiments, thepersonalized-visual-content-display system 106 extracts a set of itemfeatures from a description(s) the set of physical items 212 and a setof content features from the collection of visual content items. Thepersonalized-visual-content-display system 106 maps the set of contentfeatures for the visual content items to the set of item features toidentify a visual content item that is related or similar to the set ofphysical items 212. FIG. 6 and the corresponding discussion provideadditional detail regarding how the personalized-visual-content-displaysystem 106 retrieves the visual content item based on mapping a set ofcontent features to a set of item features in accordance with one ormore embodiments.

Additionally, or alternatively, and as mentioned above with respect tothe act 204, in some embodiments, thepersonalized-visual-content-display system 106 retrieves the visualcontent item based on analyses other than mapping a set of contentfeatures to a set of item features. For example, in one or moreembodiments, the personalized-visual-content-display system 106determines characteristics of the person 214 based on the set ofphysical items 212. As mentioned previously with respect to the act 204,in some embodiments, the personalized-visual-content-display system 106utilizes the descriptions of the physical items to determinecharacteristics of the person. FIG. 7 and the corresponding discussionprovide additional detail with respect to thepersonalized-visual-content-display system 106 determiningcharacteristics of the person in accordance with one or moreembodiments. In further embodiments, thepersonalized-visual-content-display system 106 determines the visualcontent item based on identifying commonly selected physical items basedon a co-selected-nodal graph. FIG. 8 and the corresponding discussionprovide additional detail regarding thepersonalized-visual-content-display system 106 accessing aco-selected-nodal graph in accordance with one or more embodiments.

As suggested above, the personalized-visual-content-display system 106performs at least two tasks to deliver personalized visual content itemsto a person within a physical space: (i) thepersonalized-visual-content-display system 106 selects a visual contentitem that is tailored to the person based on the set of physical items,and (ii) the personalized-visual-content-display system 106 identifies adigital-content-display device by which to display the visual contentitem based on a projected location of the person. FIG. 3 illustrates anexample physical space 300 within which thepersonalized-visual-content-display system 106 performs theabove-mentioned tasks. As illustrated in FIG. 3, the physical space 300comprises a store that contains physical items (e.g., physical items A,B, C, D, E, and F), a person 314, and a digital-content-display device312.

As illustrated in FIG. 3, the personalized-visual-content-display system106 identifies a set of physical items 308 selected by the person 314within the physical space 300. In particular, thepersonalized-visual-content-display system 106 utilizes data received byvarious sensors within the physical space 300 to identify the set ofphysical items 308 selected one by one (or in groups) by the person 314.

FIG. 3 illustrates the identified set of physical items 308 at differenttimes. For instance, at the time t1, thepersonalized-visual-content-display system 106 determines that the setof physical items 308 a comprises a physical item A. At the time t2, thepersonalized-visual-content-display system 106 determines that the setof physical items 308 b includes the physical items A, B, and C. At thetime t3, the set of physical items 308 c includes the physical items A,B, C, D, E, and F.

In one or more embodiments, the personalized-visual-content-displaysystem 106 determines the set of physical items 308 based on determinedlocations of individual physical items. In particular, thepersonalized-visual-content-display system 106 accesses sensor data todetermine locations of individual physical items. In one embodiment, thepersonalized-visual-content-display system 106 determines that aphysical item has been selected by the person 314—and thus belongs tothe set of physical items 308—based on the location of the physical itembeing within a threshold distance of the set of physical items 308.

Additionally, or alternatively, the personalized-visual-content-displaysystem 106 determines that a physical item has been selected by theperson 314—and thus add the physical item to the set of physical items308—based on determining that the physical item is moving together withone or more physical items within the set of physical items. Forinstance, the personalized-visual-content-display system 106 determinesthat the movement of a physical item is similar to the movement ofphysical items within the set of physical items 308. FIGS. 5A-5B and thecorresponding discussion provide additional detail regarding sensorsthat the personalized-visual-content-display system 106 utilizes todetermine physical items within the set of physical items 308.

Although FIG. 3 illustrates the addition of physical items to the set ofphysical items 308, the personalized-visual-content-display system 106also dynamically responds to removing physical items from the set ofphysical items. In particular, the personalized-visual-content-displaysystem 106 dynamically adjusts its analysis for selecting the visualcontent item in real time based on physical items within the set ofphysical items 308. In one example, thepersonalized-visual-content-display system 106 determines that thephysical item B is stationary at the location 302 c while the physicalitems A and C continue moving toward the location 302 d.

Based on identifying a divergence between locations of physical itemswithin a set of physical items, the personalized-visual-content-displaysystem 106 modifies the physical items within the set of physical items.For example, in one or more embodiments, thepersonalized-visual-content-display system 106 automatically determinesthat physical items that have continued in motion still belong to theset of physical items while physical items that have stopped movementare removed from the set of physical items. Additionally, oralternatively, in some embodiments, thepersonalized-visual-content-display system 106 determines to recognize asecond set of physical items based on a divergence in location ofphysical items within a set of physical items.

As mentioned, the personalized-visual-content-display system 106determines the set of physical items 308 in real time. Morespecifically, the personalized-visual-content-display system 106continuously monitors physical items within the set of physical items308. For example, in at least one embodiment, thepersonalized-visual-content-display system 106 continuously tracks thelocations of physical items within the set of physical items 308.Additionally, or alternatively, the personalized-visual-content-displaysystem 106 periodically monitors the set of physical items 308 atpredetermined intervals (e.g., every 5 seconds, every minute). In anycase, the personalized-visual-content-display system 106 continuouslydetermines identifiers for the physical items within the set of physicalitems 308.

As further illustrated in FIG. 3, thepersonalized-visual-content-display system 106 maintains a set database316 that indicates the physical items within identified sets of physicalitems. In some embodiments, the personalized-visual-content-displaysystem 106 maintains and stores different sets of physical itemsassociated with different people. For example, as illustrated in FIG. 3,the set database 316 includes a set #2468 associated with a first personand a set #1357 associated with a second person. Furthermore, thepersonalized-visual-content-display system 106 updates the set database316 in real time to reflect physical items that are currently part ofidentified sets of physical items for a particular person. For instance,the personalized-visual-content-display system 106 adds and/or removesphysical items associated with an entry of a particular set of physicalitems. In one example, the personalized-visual-content-display system106 updates the set database 316 to include sets of physical items thatare currently in motion within the physical space 300.

Based on determining that a set of physical items has left the physicalspace 300, for instance, the personalized-visual-content-display system106 stops updating the entry corresponding to that set of physicalitems. Furthermore, in some embodiments, based on determining that a setof physical items has stopped movement for a threshold period of time,the personalized-visual-content-display system 106 stops updating orremoves the entry corresponding to the idle set of physical items. Insome embodiments, the personalized-visual-content-display system 106accesses the set database 316 to select a visual content item 310.

As just suggested, the personalized-visual-content-display system 106selects the visual content item 310 from a collection of visual contentitems based on the set of physical items 308. As illustrated in FIG. 3,the personalized-visual-content-display system 106 analyzes the contentof the set of physical items 308 to select the visual content item 310that is specific to the person 314. For instance, in one or moreembodiments, the personalized-visual-content-display system 106 accessesthe set database 316 to identify physical items within a particular setof physical items. The personalized-visual-content-display system 106accordingly selects the visual content item 310 based on the one or morephysical items within the set of physical items.

In addition to selecting the visual content item 310, thepersonalized-visual-content-display system 106 also identifies adigital-content-display device 312 by which to display the visualcontent item 310. As illustrated in FIG. 3, and as mentioned, thepersonalized-visual-content-display system 106 determines locations ofthe person 314 and/or the set of physical items 308. Based on thedetermined locations, the personalized-visual-content-display system 106generates a projected path 306 and determines a projected location 304of the person 314 that is in proximity to the digital-content-displaydevice 312. The following paragraphs provide an overview of thepersonalized-visual-content-display system 106 determining locationsassociated with the set of physical items 308 for the purposes ofidentifying the digital-content-display device 312.

As mentioned, in some embodiments, thepersonalized-visual-content-display system 106 detects locations ofphysical items within a set of physical items. In particular, in one ormore embodiments, the personalized-visual-content-display system 106generates a projected path of the person based on detected locations atwhich the person interacted with or otherwise selected a physical itemfrom the set of physical items 308. For example, as illustrated in FIG.3, the personalized-visual-content-display system 106 detects locationsat which the person 314 selects the physical items. Thepersonalized-visual-content-display system 106 determines that theperson 314 selects the physical item A at the location 302 b, thephysical items B and C at the location 302 c, and the items D, E, and Fat the location 302 d. In some embodiments, thepersonalized-visual-content-display system 106 also tracks the locationof one or more physical items of the set of physical items 308 betweenthe locations 302 b-302 d.

As part of generating the projected path, in certain implementations,the personalized-visual-content-display system 106 tracks the locationof the person 314. By contrast, in some embodiments, thepersonalized-visual-content-display system 106 does not track thelocation of the person 314 if the person has not yet selected anyphysical items. More specifically, in one or more embodiments, thepersonalized-visual-content-display system 106 tracks the locations ofphysical items within a set of physical items and not the person 314.For instance, in at least one embodiment, thepersonalized-visual-content-display system 106 does not track thelocation of the person 314 between the times t0 and t1 because theperson 314 has not yet selected any physical items. Additionally, oralternatively, the personalized-visual-content-display system 106 tracksthe location of the person 314, even if the person has not selected anyphysical items. In one example, the personalized-visual-content-displaysystem 106 tracks the location of a container associated with the person314. In yet another example, the personalized-visual-content-displaysystem 106 analyzes visual data or other sensor data to determine thelocation of the person 314, even if the person has not yet selected aphysical item.

As further illustrated in FIG. 3, thepersonalized-visual-content-display system 106 generates the projectedpath 306. Generally, the projected path 306 comprises a route that theperson 314 is likely to travel in the future. In some embodiments, thepersonalized-visual-content-display system 106 analyzes historicallocation data in combination with the locations 302 a-302 d to determinethe projected path 306. FIG. 4 and the corresponding discussion provideadditional detail with respect to generating a projected path based onhistorical location data in accordance with one or more embodiments.

In some embodiments, and as further illustrated in FIG. 3, thepersonalized-visual-content-display system 106 generates the predictedpath 306 based on the set of physical items 308. Generally, thepersonalized-visual-content-display system 106 predicts that a personwho has selected the set of physical items 308 is more likely to take aparticular projected path. In particular, in some embodiments, thepersonalized-visual-content-display system 106 predicts one or moreadditional physical items that the person 314 is likely to select basedon the set of physical items 308. For example, based on determining thatthe set of physical items 308 includes baby clothing, thepersonalized-visual-content-display system 106 might predict a higherlikelihood that the person 314 will next go toward a baby-food sectionwithin the physical space 300.

As illustrated in FIG. 3, the personalized-visual-content-display system106 determines the projected location 304 of the person 314 within thephysical space 300 based on the projected path 306. Generally, theprojected location 304 comprises a point along the projected path 306 ata projected time. In one or more embodiments, thepersonalized-visual-content-display system 106 determines the projectedlocation 304 by selecting a point along the projected path 306 that isin proximity to a digital-content-display device. For example, in atleast one embodiment, the personalized-visual-content-display system 106identifies, as the projected location 304, the closest point to thecurrent location of the person 314 that is also in proximity to thedigital-content-display device 312.

As further illustrated in FIG. 3, thepersonalized-visual-content-display system 106 also determines aprojected time at which the personalized-visual-content-display system106 predicts the person 314 will arrive at the projected location 304.In some embodiments, the personalized-visual-content-display system 106utilizes a time period of a threshold duration to determine a projectedtime. For example, the personalized-visual-content-display system 106utilizes a predetermined travel speed for the person 314. Thepersonalized-visual-content-display system 106 assumes that the person314 will travel along the projected path at the predetermined travelspeed and accordingly determines the projected time at which the personwill arrive at the projected location 304. In other embodiments, thepersonalized-visual-content-display system 106 determines a travel speedfor the person 314 based on the locations 302 a-302 d as well as thetimes t0-t3. In one example, the personalized-visual-content-displaysystem 106 determines travel times between the locations 302 a-302 d anddetermines the projected time based on the determined travel times.

In sum, FIG. 3 illustrates an example environment in which thepersonalized-visual-content-display system 106 selects a visual contentitem to display and a digital-content-display device by which to displaythe selected visual content item in accordance with one or moreembodiments. FIG. 4 and the corresponding discussion provide additionaldetail regarding how the personalized-visual-content-display system 106determines a projected location of a person for selecting adigital-content-display device in accordance with one or moreembodiments. In particular, FIG. 4 illustrates a series of acts 400including an act 402 of detecting locations corresponding to the person,an act 404 of generating a projected path, and an act 406 of determininga projected location at which the person will arrive at a projectedtime.

As illustrated in FIG. 4, the series of acts 400 includes the act 402 ofdetecting locations corresponding to the person. Generally, thepersonalized-visual-content-display system 106 utilizes various methodsfor detecting locations corresponding to the person. In someembodiments, the personalized-visual-content-display system 106 detectslocations corresponding to the person by tracking locations ofindividual physical items or the set of physical items selected by theperson. In further embodiments, the personalized-visual-content-displaysystem 106 directly tracks the location of the person. FIG. 4illustrates various methods by which thepersonalized-visual-content-display system 106 tracks locationscorresponding to the set of physical items. For example, thepersonalized-visual-content-display system 106 may utilize a reader grid408, a wide-area monitor 410, or an active emitter 412 to detectlocations of the person. The following paragraphs provide additionaldetail regarding each of these implementations.

As illustrated in FIG. 4, in some embodiments, thepersonalized-visual-content-display system 106 utilizes the reader grid408 to detect locations corresponding to the person. In particular, thepersonalized-visual-content-display system 106 utilizes a grid ofreaders within the physical space to track the physical locations ofphysical items within the set of physical items. For example, asillustrated, the personalized-visual-content-display system 106 employsa grid comprising four readers.

In at least one embodiment, the readers comprise RFID sensors. An RFIDsensor transmits signals to an RFID tag 416 affixed to a physical item.The RFID tag 416 broadcasts identification information of the physicalitem and other data to the RFID sensor. Thepersonalized-visual-content-display system 106 tracks the location ofthe RFID tag 416 as it moves between signal boundaries of different RFIDsensors. By implementing the reader grid 408, thepersonalized-visual-content-display system 106 generates a rough path ofthe RFID tag 416 as the RFID tag travels between signal boundaries ofthe RFID sensors. As suggested by FIG. 4, thepersonalized-visual-content-display system 106 similarly detects signalsfrom and tracks the location of other RFID tags attached to physicalitems selected by the person.

In some embodiments, and as further illustrated in FIG. 4, thepersonalized-visual-content-display system 106 deploys the wide-areamonitor 410. In some cases, the wide-area monitor 410 comprises a readerof higher sensitivity than the readers utilized as part of the readergrid 408. More specifically, in some embodiments, the wide-area monitor410 comprises a highly sensitive fixed reader that includes an array ofantennas. In contrast to the reader grid 408 that thepersonalized-visual-content-display system 106 utilizes to determine arough path of a tag depending on the RFID sensor that captures signalsfrom the RFID tag, the personalized-visual-content-display system 106determines 2-dimensional coordinates (e.g., x, y coordinates) of a tag418 by utilizing the wide-area monitor 410. In one example, thewide-area monitor 410 comprises fixed gateway readers, such as, thexArray made by Impinj® described in Mike Lenehan, “xArray GatewayProduct Brief/Datasheet,”support.impinj.com/hc/en-us/articles/202755688-xArray-Gateway-Product-Brief-Datasheet(Jun. 5, 2020).

Additionally, or alternatively, the personalized-visual-content-displaysystem 106 tracks locations corresponding to the set of physical itemsby utilizing the active emitter 412. Generally, an active emitter 412comprises an active signal emitter. For example, in some embodiments,the active emitter 412 comprises a UWB transmitter 420 associated withthe set of physical items. In one example, the UWB transmitter 420 isaffixed to a container, such as a shopping cart or basket utilized bythe person. The UWB transmitter 420 emits signals received by beacons422 a-422 b. Based on the signals received by the beacons 422 a-422 b,the personalized-visual-content-display system 106 determines locationsassociated with the set of physical items. For example, in one or moreembodiments, the UWB transmitter 420 and the beacons 422 a-422 bcomprise transmitters and beacons described in Zoltan Koppanyi, et al.,“Performance Analysis of UWB Technology for Indoor Positioning,”Proceedings of the 2014 International Technical meeting of The Instituteof Navigation, San Diego, Calif., January 2014, pp. 154-165, the entirecontents of which are hereby incorporated by reference.

As further illustrated in FIG. 4, thepersonalized-visual-content-display system 106 performs the act 404 ofgenerating a projected path. Generally, thepersonalized-visual-content-display system 106 generates a projectedpath 424 based on detected locations at which the person interacted withor otherwise selected a physical item from the set of physical items. Inparticular, the personalized-visual-content-display system 106 generatesthe projected path based on locations corresponding to theelectromagnetic signals from the tags associated with the set ofphysical items. As mentioned previously, in some embodiments, thepersonalized-visual-content-display system 106 generates the projectedpath based on previous locations of one or more physical items of theset of physical items as indicated by the electromagnetic signals.Additionally, or alternatively, the personalized-visual-content-displaysystem 106 generates the path based on predicting one or more additionalphysical items that the person is likely to select based on the set ofphysical items.

As illustrated in FIG. 4, in some embodiments, thepersonalized-visual-content-display system 106 generates the projectedpath based on previous locations by accessing historical location data422. In particular, historical location data comprises historicallocations associated with historical persons within the physical space.For example, the personalized-visual-content-display system 106 storeshistorical location data that includes historical paths taken byhistorical persons. Generally, the personalized-visual-content-displaysystem 106 generates the projected path for a person 432 based ontrajectories of similar historical persons 434.

In one or more embodiments, the personalized-visual-content-displaysystem 106 predicts that the person 432 will travel a projected path 424that is the same as or similar to historical paths 428 traveled byhistorical persons. To illustrate, in some embodiments, thepersonalized-visual-content-display system 106 determines locations 430corresponding to the set of physical items. In particular, the locations430 together indicate where the person 432 has traveled within thephysical location over a period of time. Thepersonalized-visual-content-display system 106 identifies, from thehistorical location data 422, historical locations 426 associated withthe historical persons 434 within the physical space. As illustrated inFIG. 4, the personalized-visual-content-display system 106 identifiesthe historical locations 426 that are the same as the locations 430. Thepersonalized-visual-content-display system 106 determines historicalpaths 428 corresponding to the historical locations. In someembodiments, the personalized-visual-content-display system 106determines the projected path 424 based on the most frequently takenhistorical paths 428.

In some embodiments, the personalized-visual-content-display system 106relies on analyses beside the historical paths 428 to determine theprojected path 424. In one example, thepersonalized-visual-content-display system 106 predicts that the personwill travel the projected path 424 utilizing deep learning algorithms.For example, in one or more embodiments, thepersonalized-visual-content-display system 106 employstrajectory-determination techniques, such as trajectory datapreprocessing, trajectory clustering, trajectory pattern mining,trajectory segmentation, and/or trajectory representation. The abovelisted techniques are described in Ruizhi Wu, et al., “LocationPrediction on Trajectory Data: A Review,” Big Data Mining and Analytics,vol. 1, no. 2, pp. 108-127, June 2018, the entire contents of which arehereby incorporated by reference.

In some embodiments, the personalized-visual-content-display system 106dynamically updates the projected path 424 as the person 432 and/or theset of physical items move through the physical space. In particular,based on determining that the person is at a new location that divergesfrom an initial projected path, the personalized-visual-content-displaysystem 106 generates an updated projected path based on the newlocation. More specifically, the personalized-visual-content-displaysystem 106 updates the projected path by analyzing historical locationdata comprising a historical location that corresponds to the newlocation.

As further illustrated in FIG. 4, thepersonalized-visual-content-display system 106 performs the act 406 ofdetermining a projected location at which the person will arrive at aprojected time. As mentioned previously, thepersonalized-visual-content-display system 106 identifies a projectedlocation 440 along the projected path 424. In some embodiments, theprojected location 440 comprises a location where the projected path 424meets or is within a threshold distance of a digital-content-displaydevice.

Additionally, and as illustrated in FIG. 4, thepersonalized-visual-content-display system 106 determines a projectedtime (t2) at which the person 432 will arrive at the projected location440. As mentioned previously, in some embodiments, thepersonalized-visual-content-display system 106 determines the projectedtime (t2) by adding a threshold duration to a current time (t1).Additionally, or alternatively, the personalized-visual-content-displaysystem 106 determines travel times based on locations corresponding tothe set of physical items and travel times between the locations. Forexample, in one or more embodiments, thepersonalized-visual-content-display system 106 determines an averagetravel speed for the person 432 based on the locations and the recordedtravel times. Alternatively, the personalized-visual-content-displaysystem 106 identifies an average travel speed for people generally. Thepersonalized-visual-content-display system 106 then determines theprojected time (t2) based on the distance between the projected location440 and a current location 442 of the person 432 and the average travelspeed for the person 432.

FIG. 4 and the corresponding discussion describe how thepersonalized-visual-content-display system 106 identifies adigital-content-display device based on generating a projected path forthe person and/or the set of physical items. FIGS. 5A-8 and thecorresponding paragraphs provide additional detail regarding how thepersonalized-visual-content-display system 106 selects a visual contentitem to display via the identified digital-content-display device. Inparticular, FIGS. 5A-5B illustrate various methods by which thepersonalized-visual-content-display system 106 identifies the set ofphysical items. FIGS. 6-8 illustrate various methods by which thepersonalized-visual-content-display system 106 selects a visual contentitem based on the set of physical items.

FIG. 5A illustrates an example physical space 500 comprising varioussensors for identifying the set of physical items. In particular, thephysical space 500 includes an electromagnetic sensor 502,electromagnetic tags 504 a-504 c affixed to physical items 510 a-510 c,and an item-holding structure 512.

In some embodiments, the personalized-visual-content-display system 106determines the physical items 510 a-510 c within the set of physicalitems based on electromagnetic signals emitted by the electromagnetictags 504 a-504 c affixed to the physical items 510 a-510 c. Inparticular, and as illustrated in FIG. 5A, the electromagnetic sensor502 receives the electromagnetic signals emitted by the electromagnetictags 504 a-504 c. The electromagnetic signals include identifiers for(or otherwise indicate the identities of) the corresponding physicalitems 510 a-510 c. In one or more embodiments, the electromagnetic tags504 a-504 c comprise RFID tags.

FIG. 5A illustrates the electromagnetic sensor 502 located on theceiling of the physical space 500. As mentioned previously, in someembodiments, the physical space 500 includes an array of electromagneticsensors. The electromagnetic sensor 502 may be located in any locationsuitable for capturing signals emitted by the electromagnetic tags 504a-504 c. In some embodiments, the container 508 is associated with acontainer electromagnetic sensor. In such embodiments, the containerelectromagnetic sensor receives electromagnetic signals from theelectromagnetic tags 504 a-504 c located within the container 508. Thecontainer electromagnetic sensor further emits signals to theelectromagnetic sensor 502 including identifiers for (or to otherwiseindicate the identities of) the physical items 510 a-510 c locatedwithin the container 508.

As further illustrated in FIG. 5A, the physical space 500 includes theitem-holding structure 512. In some embodiments, the item-holdingstructure 512 is associated with one or more sensors that indicate thepresence of physical items located within or on the item-holdingstructure 512. For example, in one or more embodiments, the item-holdingstructure 512 includes an electromagnetic sensor that determines thepresence of physical items within the item-holding structure 512 basedon signals from affixed electromagnetic tags. In further embodiments,the item-holding structure 512 includes other types of sensors, such aspressure sensors and/or cameras that detect the presence of physicalitems within the item-holding structure 512. Sensors associated with theitem-holding structure 512 communicate with thepersonalized-visual-content-display system 106 to indicate whether aphysical item has been removed from the item-holding structure 512. Asillustrated in FIG. 5A, for example, the item-holding structure 512comprises a shelving unit with smart shelving features and sensors. Butthe personalized-visual-content-display system 106 can use any suitableitem-holding structure, including containers, hangers, racks, etc.

The physical space 500 illustrated in FIG. 5A also includes a camera514. The camera 514 captures visual data relating to a set of physicalitems. Generally, the camera 514 captures visual data that trackslocations and movement of physical items within the physical space 500.In some embodiments, the personalized-visual-content-display system 106analyzes image and/or video data captured by the camera 514. Morespecifically, the personalized-visual-content-display system 106utilizes computer vision to identify physical items within the imageand/or video data.

In some embodiments, the personalized-visual-content-display system 106utilizes computer vision to determine the location of the person 506. Inone example, the personalized-visual-content-display system 106 utilizesvisual data from the camera 514 in conjunction with other sensors withinthe physical space 500 to identify a set of physical items. Forinstance, in one or more embodiments, thepersonalized-visual-content-display system 106 determines, based onvisual data captured by the camera 514, that the person 506 is locatedwithin a threshold proximity of the item-holding structure 512. Based onreceiving signals from the item-holding structure 512 that particularphysical items have left the item-holding structure 512, thepersonalized-visual-content-display system 106 determines that theperson 506 selected the particular physical items.

As described previously, in certain implementations, thepersonalized-visual-content-display system 106 determines the set ofphysical items by identifying co-located physical items that movetogether through the physical space. For instance, in some embodiments,the personalized-visual-content-display system 106 identifies a set ofphysical items by determining that one or more physical items areco-located or located within a threshold distance of each other. In onesuch case, the personalized-visual-content-display system 106 determinesthat physical items located within a threshold distance of two feet (orsome other threshold distance) of each other are co-located and thusbelong to a single set of physical items. In further embodiments, thepersonalized-visual-content-display system 106 determines the set ofphysical items by identifying items of interest. Generally, items ofinterest comprise physical items with which a person has interacted buthas not necessarily added to the set of physical items. In one example,an item of interest comprises an item that the person picks up butleaves. FIG. 5B and the corresponding discussion detail how thepersonalized-visual-content-display system 106 identifies an item ofinterest in accordance with one or more embodiments.

As further suggested above, in one or more embodiments, thepersonalized-visual-content-display system 106 determines the set ofphysical items based on one or more physical items that are within agroup at a given time. In particular, thepersonalized-visual-content-display system 106 adds a physical item tothe set of physical items based on determining that the person hasselected and traveled with the physical item for at least a thresholddistance or a threshold amount of time. In one example, based ondetermining that the location of a particular physical item divergesfrom locations corresponding to remaining physical items within the setof physical items, the personalized-visual-content-display system 106removes the particular physical item from the set of physical items.

By contrast, in further embodiments, thepersonalized-visual-content-display system 106 adds all physical itemswith which the person has interacted to the set of physical items,regardless of whether the physical items are added to the group ofphysical items. In one example, the personalized-visual-content-displaysystem 106 adds a physical item to the set of physical items, even ifthe person simply picked up, viewed, and set down the physical item. Asillustrated in FIG. 5B, for instance, thepersonalized-visual-content-display system 106 determines a physicalitem of a set of physical items by determining one or more motion events526 corresponding to a physical item 524. Additionally, oralternatively, the personalized-visual-content-display system 106determines that a physical item belongs within a set of physical itemsbased on areas 528 to which the physical item 524 has traveled.

In some embodiments, the personalized-visual-content-display system 106determines motion events 526 corresponding to the physical item 524.Generally, the motion events 526 comprise occurrences marked my movementof a physical item. In particular, a motion event comprises an instancein which a person interacts with a physical item. Examples of motionevents comprise a person touching, picking up, moving, or setting down aphysical item. For instance, as illustrated in FIG. 5B, thepersonalized-visual-content-display system 106 identifies the motionevents 526 associated with the physical item 524 at different times (t1and t2). The personalized-visual-content-display system 106 determinesthat, at t1, the physical item 524 was picked up, and at t2, thephysical item 524 was stationary.

In one or more embodiments, the personalized-visual-content-displaysystem 106 analyzes electromagnetic signals emitted by tags affixed tophysical items to determine the motion events 526. In one example, thepersonalized-visual-content-display system 106 determines the motionevents 526 based on RFID signals. For instance, thepersonalized-visual-content-display system 106 receives, from an RFIDreader, signal measurements reflected by an RFID tag affixed to thephysical item 524. More specifically, the signal measurements comprisesignal power, phase angle, and frequency shift. Thepersonalized-visual-content-display system 106 determines the motionevents 526 based on the signal measurements. Thepersonalized-visual-content-display system 106 further identifies thephysical item 524 as an item of interest based on the one or more motionevents. For example, in one or more embodiments, thepersonalized-visual-content-display system 106 identifies items ofinterest based on the motion events 526 as described in Hanchuan Li, etal., “IDSense: A Human Object Interaction Detection System Based onPassive UHF RFID,” CHI '15: Proceedings of the 33^(rd) Annual ACMConference on Human Factors in Computing Systems, pp. 2555-256,https://doi.org/10.1145/2702123.2702178, (April 2015), the entirecontents of which are hereby incorporated by reference.

Furthermore, and as illustrated in FIG. 5B, in some embodiments, thepersonalized-visual-content-display system 106 determines physical itemswithin the set of physical items based on areas 528 within the physicalspace to which the physical items have been moved. Generally, thepersonalized-visual-content-display system 106 determines that aphysical item belongs to a set of physical items based on the movementof the physical item into a particular area within the physical space.For example, the personalized-visual-content-display system 106determines that the physical item 524 is an item of interest based ondetermining that the person moved the physical item 524 from a rack area522 to a fitting room area 520. In this example, thepersonalized-visual-content-display system 106 determines that physicalitems that have been moved into the fitting room area 520 are associatedwith a higher likelihood of being items of interest to the person.

Accordingly, in some embodiments, thepersonalized-visual-content-display system 106 determines that aphysical item is not a part of a set of physical items based on theperson excluding the physical item from the person's movement into aparticular area. For example, and as illustrated in FIG. 5B, thepersonalized-visual-content-display system 106 determines that aphysical item is not part of a set of physical items if the physicalitem is left in the rack area 522 and is not brought along with otherphysical items into the fitting room area 520. To illustrate, thepersonalized-visual-content-display system 106 determines that theperson moved a first physical item to an area (e.g., the fitting roomarea 520) within the physical space while also determining that theperson did not move a second physical item into the area. Based on thesedeterminations, the personalized-visual-content-display system 106further determines that the first physical item is part of the set ofphysical items while the second physical item is not part of the set ofphysical items.

As discussed, the personalized-visual-content-display system 106 selectsa visual content item to display via a digital-content-display devicebased on a set of physical items. FIGS. 5A-5B illustrate various methodsby which the personalized-visual-content-display system 106 identifiesphysical items within the set of physical items in accordance with oneor more embodiments. FIGS. 6-8 provide example methods by which thepersonalized-visual-content-display system 106 selects visual contentitems based on the set of physical items in accordance with one or moreembodiments. By way of overview, FIG. 6 illustrates thepersonalized-visual-content-display system 106 selecting visual contentitems based on item embeddings and content embeddings in accordance withone or more embodiments. FIG. 7 illustrates thepersonalized-visual-content-display system 106 retrieving a visualcontent item based on characteristics of the person in accordance withone or more embodiments. FIG. 8 illustrates thepersonalized-visual-content-display system 106 accessing and analyzing aco-selected-nodal graph in accordance with one or more embodiments.

FIG. 6 illustrates a series of acts 600 for determining a measure ofsimilarity between a set of physical items and visual content itemsbased on item and content embeddings. In some embodiments, thepersonalized-visual-content-display system 106 selects a visual contentitem from a collection of visual content items by performing aword-embedding match or word-embedding comparison between thedescriptions of physical items and descriptions of images. The series ofacts 600 includes an act 602 of identifying a set of descriptions for aset of physical items, an act 604 of generating item embeddings, an act606 of generating content embeddings, and an act 608 of determining ameasure of similarity.

As illustrated in FIG. 6, the personalized-visual-content-display system106 performs the act 602 of identifying a set of descriptions for a setof physical items. As part of determining the set of physical items, thepersonalized-visual-content-display system 106 accesses identifiers ofphysical items in the set of physical items. Thepersonalized-visual-content-display system 106 utilizes the identifiersof the physical items to access descriptions of the physical itemswithin the set of physical items. For example, in at least oneembodiment, the personalized-visual-content-display system 106 utilizesa physical item identifier to look up a description for the physicalitem within a physical item database. As illustrated in FIG. 6, thepersonalized-visual-content-display system 106 determines that thedescription for the physical item associated with the identifier “A” is“Plaid blouse, women's, size M.” In some embodiments, thepersonalized-visual-content-display system 106 aggregates orconcatenates the description for each physical item in a set of physicalitems into a single description.

As further illustrated in FIG. 6, thepersonalized-visual-content-display system 106 performs the act 604 ofgenerating item embeddings. Generally, thepersonalized-visual-content-display system 106 extracts, utilizing alanguage-embedding model 610, a set of item features from the set ofdescriptions. In some embodiments, thepersonalized-visual-content-display system 106 generates individual itemfeatures 612 a-612 b, each corresponding to a physical item within theset of physical items. For example, as illustrated in FIG. 6, theindividual item features 612 a-612 b comprise individual feature vectorscorresponding to the physical items A and B, respectively. In certainembodiments, the personalized-visual-content-display system 106generates a combined item feature 614 corresponding to aggregated orconcatenated descriptions of all physical items within the set ofphysical items. For example, as illustrated in FIG. 6, thepersonalized-visual-content-display system 106 aggregates orconcatenates the descriptions corresponding to the physical items A andB. The personalized-visual-content-display system 106 utilizes thelanguage-embedding model 610 to extract the combined item feature 614from the descriptions of the combined physical items A and B.

Generally, the language-embedding model 610 illustrated in FIG. 6comprises one or more language models that map words or phrases tofeatures such as vectors. The language-embedding model 610 comprises anysuitable model utilized to generate item features. For example, in oneor more embodiments, the language-embedding model comprises a Word2Vecmodel. Generally, a Word2Vec model may be implemented as a neuralnetwork trained to reconstruct usage contexts of features as featureembeddings.

Alternatively, in some embodiments, thepersonalized-visual-content-display system 106 utilizes a bi-directionallong-short-term-memory (Bi-LSTM) network or a gated-recurrent-unit (GRU)network as the language-embedding model 610 to extract word embeddingsor language embeddings from descriptions, such as Global Vectors (GloVe)embeddings.

Additionally, in certain implementations, thepersonalized-visual-content-display system 106 further passes such wordembeddings or language embeddings through repeated units of a fullyconnected (FC) layer with dropout, a rectified linear units (ReLU)activation layer, and a L2-norm (L2 Norm) layer before generating itemembeddings, such that the item beddings share a common-embedding spaceor dimensionality as the content embeddings.

As further illustrated in FIG. 6, thepersonalized-visual-content-display system 106 performs the act 606 ofgenerating content embeddings. In particular, thepersonalized-visual-content-display system 106 generates, utilizing animage-embedding model 616, content features 618 a-618 b for visualcontent items 620 a-620 b. As illustrated in FIG. 6, the contentfeatures 618 a-618 b comprise feature vectors (e.g., a series ofnumbers). In some embodiments, the image-embedding model 616 accesses(and extracts the content features 618 a-618 b from) metadata associatedwith the visual content items 620 a-620 b. The metadata comprisestextual descriptions of the visual content items 620 a-620 b. Thus, insome embodiments, the image-embedding model 616 comprises one or morelanguage models that map words or phrases to features, such as vectors.In one example, the image-embedding model 616 comprises a Word2Vecmodel.

By contrast, in certain embodiments, thepersonalized-visual-content-display system 106 extracts the contentfeatures 618 a-618 b from the visual content items 620 a-620 b using analternative image embedding model. For example, thepersonalized-visual-content-display system 106 can use an ImageNet, aDenseNet, or other ResNet as the image embedding model 616 to extractthe content features 618 a-618 b from the visual content items 620 a-620b.

As further shown in FIG. 6, the series of acts 600 also includes the act608 of determining a measure of similarity. In particular, in one ormore embodiments, the personalized-visual-content-display system 106maps the content features 618 a-618 b to the individual item features612 a-612 b and/or the combined item feature 614 in a common embeddingspace. For example, as illustrated in FIG. 6, thepersonalized-visual-content-display system 106 maps content embeddings626 a-626 b (corresponding to the content features 618 a-618 b,respectively) to the individual item embeddings 624 a-624 b(corresponding to the individual item features 612 a-612 b,respectively) within a common embedding space 622 a. Thepersonalized-visual-content-display system 106 further determinesmeasures of similarity among the individual item embeddings 624 a-624 band the content embeddings 626 a-626 b based on embedding distances orother methods.

Indeed, the personalized-visual-content-display system 106 can deployvarious methods (or algorithms) to determine measures of similarity. Inaddition to embedding-space distances, for example, thepersonalized-visual-content-display system 106 utilizes cosinesimilarity calculations, k-nearest neighbor calculations, and/orclustering techniques to determine distances between the individual itemembeddings 624 a-624 b and the content embeddings 626 a-626 b. In one ormore embodiments, the personalized-visual-content-display system 106selects a visual content item based on the content embedding of thecontent embeddings 626 a-626 b with the lowest average distance to theindividual item embeddings 624 a-624 b.

As illustrated in FIG. 6, the personalized-visual-content-display system106 selects the visual content item corresponding to the contentembedding 626 a based on determining that the content embedding 626 a islocated the shortest average distance from the individual itemembeddings 624 a-624 b. In some implementations, thepersonalized-visual-content-display system 106 selects more than onevisual content item by identifying multiple content embeddings within athreshold distance of the item embedding(s). Additionally, oralternatively, the personalized-visual-content-display system 106selects more than one visual content item by identifying a thresholdnumber of the closest content embeddings.

In the alternative to mapping individual item embeddings to one or morecontent embedding, in some embodiments, thepersonalized-visual-content-display system 106 maps the contentembeddings 626 a-626 b to a combined item embedding 628 (correspondingto the combined item feature 614) within a common embedding space 622 b.The personalized-visual-content-display system 106 can determinedistances between the combined item embedding 628 and the contentembeddings 626 a-626 b utilizing the methods (or algorithms) describedabove. For example, the personalized-visual-content-display system 106performs cosine similarity calculations between the combined itemembedding 628 and the content embeddings 626 a-626 b. Thepersonalized-visual-content-display system 106 selects the visualcontent item based on identifying a content embedding located theshortest distance from the combined item embedding 628 within the commonembedding space 622 b.

In one or more embodiments, the personalized-visual-content-displaysystem 106 selects a visual content item based on determiningcharacteristics of the person. FIG. 7 illustrates thepersonalized-visual-content-display system 106 performing a series ofacts 700 for retrieving a visual content item based on characteristicsof the person. In particular, the series of acts 700 includes an act 702of identifying a set of descriptions for a set of physical items, an act704 of determining characteristics of the person, an act 706 ofaccessing metadata of visual content items, and an act 708 of retrievingthe visual content item.

As illustrated in FIG. 7, the series of acts 700 includes the act 702 ofidentifying a set of descriptions for a set of physical items. Aspreviously described, in certain implementations, thepersonalized-visual-content-display system 106 accesses descriptionscorresponding to identifiers of each physical item within the set ofphysical items.

As further illustrated by FIG. 7, the series of acts 700 furtherincludes the act 704 of determining characteristics of the person.Generally, the personalized-visual-content-display system 106 infers,based on the set of physical items, characteristics or attributes of aperson based on the set of physical items selected by the person. Insome embodiments, the personalized-visual-content-display system 106infers at least one of an age, gender, spend level, size, or shoppingpurpose based on the set of physical items. In further embodiments, thepersonalized-visual-content-display system 106 infers additionalattributes of the person, such as a body type and other characteristics.As illustrated in FIG. 7, for instance, thepersonalized-visual-content-display system 106 determines, based on thedescriptions associated with the physical items A-B, the age, gender,spend level, size, and shopping purpose for the person.

To illustrate, in one or more embodiments, as part of performing the act704 of determining characteristics of the person, thepersonalized-visual-content-display system 106 determinescharacteristics and corresponding confidence scores based on thedescription(s) of physical items. For instance, thepersonalized-visual-content-display system 106 determinescharacteristics of the person from the description(s) of the physicalitems—or from item features extracted from the description(s)—andgenerates confidence scores for such characteristics. In some instance,the personalized-visual-content-display system 106 applies aclassification algorithm to the description(s) of the physical items.Alternatively, in some cases, the personalized-visual-content-displaysystem 106 initially utilizes a language embedding model to extractindividual item features and/or the combined item features from the setof descriptions. The personalized-visual-content-display system 106further analyzes the individual item features and/or the combined itemfeatures to determine the characteristics of the person (e.g., age,gender, spend level, size, shopping purpose) and correspondingconfidence scores.

In one or more embodiments, the personalized-visual-content-displaysystem 106 utilizes classification algorithms to determine suchcharacteristics of the person based on either the description(s) of thephysical items or the item features extracted from the description(s) bythe language embedding model. For instance, thepersonalized-visual-content-display system 106 can pass thedescription(s) or the item features through a Support Vector Machine(SVM), a Long Short Term Memory (LSTM) network (e.g., Bi-LSTM), aConvolutional Neural Network (CNN), or other classification algorithmtrained on natural language, bag of words, or textual embeddings (e.g.,word embeddings) to classify the description(s) or the item featuresextracted from the description(s) and determine characteristics of theperson along with corresponding confidence scores.

As indicated above, the confidence scores indicate the likelihood thatthe characteristic is accurately associated with the person. In oneexample, the personalized-visual-content-display system 106 determines(i) a “female” characteristic based on the description(s) or theextracted item features and (ii) a 0.85 confidence score indicating thelikelihood that the person is a female. Furthermore, thepersonalized-visual-content-display system 106 associates one or more ofa succession of age ranges with the person based on the description(s)or the extracted item features. For example, thepersonalized-visual-content-display system 106 may associate one of theage ranges 25-40, 30-35, 27-33, etc. with a particular person. In somecases, the personalized-visual-content-display system 106 furthergenerates a confidence score for each age range. Thepersonalized-visual-content-display system 106 can infer which age rangethe person belongs based on such confidence scores.

As further shown in FIG. 7, the personalized-visual-content-displaysystem 106 further performs the act 706 of accessing metadata of visualcontent items. In particular, the personalized-visual-content-displaysystem 106 accesses text descriptions of visual content items 710 a-710b within a collection of visual content items. Alternatively, thepersonalized-visual-content-display system 106 accesses keywords orcodes as metadata corresponding to the visual content items 710 a-710 b.

After accessing the metadata, as further illustrated in FIG. 7, thepersonalized-visual-content-display system 106 performs the act 708 ofretrieving the visual content item. Generally, thepersonalized-visual-content-display system 106 selects the visualcontent item from the visual content items 710 a-710 b of the collectionof visual content items. In some embodiments, thepersonalized-visual-content-display system 106 utilizes a hash table 712that maps characteristics of the person with word descriptors extractedfrom the metadata of the visual content items.

As part of building the hash table 712, in certain implementations, thepersonalized-visual-content-display system 106 determines hash keys forthe characteristics and the word descriptors. Generally, thepersonalized-visual-content-display system 106 utilizes a hash functionto convert input data (i.e., the characteristics and/or the worddescriptors) into a hash key. In particular, in some embodiments, a hashkey comprises a compressed numerical value. Furthermore, in one or moreembodiments, the hash key comprises a numerical value of a fixed length.To illustrate, in some embodiments, thepersonalized-visual-content-display system 106 divides input datacomprising the characteristics or the word descriptors into fixed sizeddata blocks. The personalized-visual-content-display system 106 inputsthe data blocks into the hash function to generate the hash key.

To generate such hash keys, the personalized-visual-content-displaysystem 106 may utilize various types of hash functions. For example, inone or more embodiments, the hash function comprises a message digestalgorithm, a secure hash algorithm (SHA), a race integrity evaluationmessage digest, a whirlpool, or a Rivest-Shamier-Adleman (RSA)algorithm, and others.

To illustrate, as part of the act 708, thepersonalized-visual-content-display system 106 creates the hash table712 that matches various characteristics or attributes with particulardescriptors from the metadata. In some embodiments, thepersonalized-visual-content-display system 106 identifies several worddescriptors corresponding to a single person. For example, thepersonalized-visual-content-display system 106 looks up word descriptorsassociated with each of the characteristics of the person. For instance,based on determining the person is likely female and approximately 31years old—or within a particular age range—based on confidence scores,the personalized-visual-content-display system 106 identifies the worddescriptors “woman, women, girl.” As suggested above, thepersonalized-visual-content-display system 106 can use or look up anysuitable word descriptors. Instead of “woman, women, girl,” forinstance, the personalized-visual-content-display system 106 mayidentify the word descriptors “female,” “adult,” and “woman” based ondetermining the person is likely female and a particular age or within aparticular age range.

In one embodiment, the personalized-visual-content-display system 106identifies candidate visual content items 714 that correspond to theidentified word descriptors based on the searched characteristics. Inparticular, the personalized-visual-content-display system 106 filtersthe collection of visual content items using the identified worddescriptors. In one or more embodiments, thepersonalized-visual-content-display system 106 ranks the candidatevisual content items 714 based on relevance. For example, thepersonalized-visual-content-display system 106 ranks the candidatevisual content items 714 based on the number of relevant worddescriptors. The personalized-visual-content-display system 106 furtherselects the visual content item 710 a from the candidate visual contentitems 714. For example, in one or more embodiments, thepersonalized-visual-content-display system 106 selects the highestranked visual content item of the candidate visual content items 714.

In addition to the methods described above, in some embodiments, thepersonalized-visual-content-display system 106 also selects a visualcontent item for display based on a co-selected-nodal graph. FIG. 8illustrates a series of acts 800 by which thepersonalized-visual-content-display system 106 selects a visual contentitem based on a co-selected-nodal graph in accordance with one or moreembodiments. As illustrated in FIG. 8, the series of acts 800 includesan act 802 of accessing a co-selected-nodal graph, an act 804 ofdetermining a commonly selected physical item, and an act 806 ofselecting a visual content item.

The series of acts 800 includes the act 802 of accessing aco-selected-nodal graph. Generally, the co-selected-nodal graphillustrated in FIG. 8 indicates the likelihood, given the person alreadyselected a physical item, that the person will select other particularphysical items. For instance, the node 808 represents a physical item Athat the person has already selected. Nodes 810 b-810 d correspond tophysical items B-D, respectively. In one example, the co-selected-nodalgraph comprises a co-bought graph that indicates the likelihood that theperson will buy particular items together with the physical item A. In afurther example, the co-selected-nodal graph comprises a co-view graphthat indicates the likelihood that the person will view particular itemstogether with the physical item A.

In some embodiments, the personalized-visual-content-display system 106calculates a score for some nodes within the co-selected-nodal graph.For example, in one or more embodiments, thepersonalized-visual-content-display system 106 generates scores for thenodes 810 b-810 d. More specifically, the scores represent a relevanceof each node to the physical item A represented by the node 808.

To calculate the relevance score for an individual node, thepersonalized-visual-content-display system 106 determines valuesassociated with some or all of the nodes (within the co-selected-nodalgraph) as part of an algorithm. In some such embodiments, thepersonalized-visual-content-display system 106 utilizes an algorithmthat can be applied to the nodes and edges of the co-selected-nodalgraph. For example, in some embodiments, thepersonalized-visual-content-display system 106 calculates and/or assignsa weight for each edge that connects one node to another node within theco-selected-nodal graph (or portion of the co-selected-nodal graph) as afunction of one or more factors. Based on the determined weight for eachedge within the co-selected-nodal graph (or portion of theco-selected-nodal graph), the personalized-visual-content-display system106 generates a probability distribution indicating the likelihood thata person would navigate from one node to another node (i.e., select acorresponding physical item) within the co-selected-nodal graph. To doso, the personalized-visual-content-display system 106 runs severaliterations of a random walk within the co-selected-nodal graph (orportion of the co-selected-nodal graph) to calculate the probabilitythat a person would navigate from one node to another node based on thedetermined weight for each edge.

A person having ordinary skill in the art will recognize, however, thatthe personalized-visual-content-display system 106 may utilize anyalgorithm suitable for analyzing co-selected-nodal graph to quantify,define, and/or rank the relationships among nodes of theco-selected-nodal graph. For example, in some embodiments, thepersonalized-visual-content-display system 106 inputs weights associatedwith some or all of the edges from a generated co-selected-nodal graphinto any algorithm that aggregates the effect of the connections amongnodes to determine an overall relevance between two specific nodes. Inparticular, in some embodiments, the personalized-visual-content-displaysystem 106 applies a DeepWalk algorithm, a PageRank random walk or amodified form of a PageRank (e.g., TrustRank) to the co-selected-nodalgraph to calculate the relevance score for nodes within a portion or allof the co-selected-nodal graph.

To illustrate, in one or more embodiments, as part of performing the act802, the personalized-visual-content-display system 106 generates aco-selected-nodal graph utilizing undirected edges. More specifically,and as illustrated, if the physical item A is frequently co-purchasedwith the items B-D, then the personalized-visual-content-display system106 generates undirected edges between the node 808 and each of thenodes 810 b-810 d. In other embodiments, thepersonalized-visual-content-display system 106 generates the undirectededges based on determining that the physical item A is frequentlyco-viewed with the physical items B-D. In some cases, physical itemsthat are co-viewed—such as one or more of the physical itemsB-D—comprise physical items that the person selects or interacts with,but not necessarily purchases, together. For instance, the physicalitems B-D may include physical items people habitually select together,but do not ultimately purchase. Examples of co-selected-nodal graphsinclude networks described in Jaewon Yang, et al. “Defining andEvaluating Network Communities based on Ground-truth,” Proceedings of2012 IEEE International Conference on Data Mining, arXiv:1205.6233(2012), the entirety of which is incorporated by reference.

As further illustrated in FIG. 8, the series of acts 800 includes theact 804 of determining a commonly selected physical item. Generally, thepersonalized-visual-content-display system 106 analyzes the probabilitydistribution generated utilizing the iterations of the random walkwithin the co-selected-nodal graph. In some embodiments, thepersonalized-visual-content-display system 106 determines the nodeassociated with the highest score or probability as the commonlyselected physical item. For instance, as illustrated in FIG. 8, thepersonalized-visual-content-display system 106 determines that thephysical item B is the commonly selected physical item 804 based on thescore associated with Item B. But thepersonalized-visual-content-display system 106 can analyze and comparescores for nodes representing other physical items within theco-selected-nodal graph consistent with the algorithms noted above(e.g., DeepWalk, PageRank).

After determining a commonly selected physical item, the series of acts800 further includes the act 806 of selecting a visual content item. Inparticular, the personalized-visual-content-display system 106 selects avisual content item that displays the commonly selected physical item.For example, and as illustrated in FIG. 8, thepersonalized-visual-content-display system 106 selects a visual contentitem 812 displaying the physical item B, which is a pair of pants.

As suggested above, in some embodiments, thepersonalized-visual-content-display system 106 updates a digital contentcampaign—such as by selecting a different visual content item from adifferent digital content campaign—in real time based on observedbehaviors of a person. FIG. 9 illustrates a series of acts 900 formodifying a digital content campaign based on behaviors of the person inaccordance with one or more embodiments. In particular, the series ofacts 900 includes an act 902 of determining an initial campaign, an act904 of analyzing behaviors of the person, and an act 906 of modifyingthe campaign.

As illustrated in FIG. 9, the personalized-visual-content-display system106 performs the act 902 of determining an initial campaign. Generally,the personalized-visual-content-display system 106 identifies a digitalcontent campaign as an initial campaign. In particular, digital contentcampaigns comprise organized assets for promoting and selling one ormore physical items. For example, a digital content campaign maycomprise digital content including visual content items. In oneembodiment, the initial campaign refers to an individual visual contentitem. In further embodiments, the initial campaign refers to a digitalcontent campaign comprising several organized assets.

In one or more embodiments, performing the act 902 comprises retrievinga visual content item for display via a particulardigital-content-display device based on a set of physical items as wellas a projected path of the person. In particular, thepersonalized-visual-content-display system 106 determines the initialcampaign for a person based on the set of physical items. Furthermore,as part of the act 902, the personalized-visual-content-display system106 determines a projected location of the person at a projected timebased on locations associated with the person and/or the set of physicalitems. As illustrated in FIG. 9, the personalized-visual-content-displaysystem 106 selects a campaign 908 a of the campaigns 908 a-908 c as theinitial campaign.

After determining an initial campaign, as further shown in FIG. 9, thepersonalized-visual-content-display system 106 performs the act 904 ofanalyzing behaviors of the person. In particular, thepersonalized-visual-content-display system 106 analyzes behaviors of theperson after the person selected items within the set of physical items.Furthermore, in some embodiments, thepersonalized-visual-content-display system 106 generates predictedbehaviors based on the set of physical items and locations of theperson. The personalized-visual-content-display system 106 analyzesbehaviors of a person 914 and determines whether the behaviors areconsistent with predicted behaviors.

To illustrate, in some embodiments, thepersonalized-visual-content-display system 106 generates the projectedpath 910 for the person 914. More specifically, thepersonalized-visual-content-display system 106 predicts that the person914 will travel along the projected path 910 to adigital-content-display device 916 displaying visual content item fromthe initial campaign. As part of performing the act 904 of analyzingbehaviors of the person, the personalized-visual-content-display system106 determines that the person 914 does not travel the projected path910 but rather a realized path 912 toward a digital-content-displaydevice 918.

As further illustrated in FIG. 9, thepersonalized-visual-content-display system 106 performs the act 906 ofmodifying the campaign. For instance, in one or more embodiments, basedon the person 914 traveling along the realized path 912 and not theprojected path 910, the personalized-visual-content-display system 106determines to modify the campaign. As illustrated in FIG. 9, thepersonalized-visual-content-display system 106 determines to select thecampaign 908 b instead of the initially selected campaign 908 a of thecampaigns 908 a-908 c.

In one example, the personalized-visual-content-display system 106performs the act 906 of modifying the campaign by selecting a campaignrelated to physical items in a proximity to the digital-content-displaydevice 918. In further embodiments, thepersonalized-visual-content-display system 106 utilizes a banditalgorithm to modify the campaign based on factors including the realizedpath 912, the location of the digital-content-display device 918, andother factors. For instance, the personalized-visual-content-displaysystem 106 utilizes multi-armed-bandit algorithms to select the campaign908 b. Examples of multi-armed-bandit algorithms utilized by thepersonalized-visual-content-display system 106 include an Epsilon-Greedyalgorithm, an upper confidence bound algorithm, and a Thompson samplingalgorithm. Example algorithms are further described in AleksandrsSlivkins, “Introduction to Multi-Armed Bandits,” arXiv:1904.0727272v5(30 Sep. 2019), the entirety of which is incorporated by reference.

As described, the personalized-visual-content-display system 106performs the act 906 before a point of sale within the physical space.In particular, the personalized-visual-content-display system 106performs the act 906 of modifying the campaign while the person travelsthrough the physical space. Additionally, or alternatively, in somecases, the personalized-visual-content-display system 106 performs theact 906 of modifying the campaign based on analyzing behaviors of theperson at a point of sale. In particular, thepersonalized-visual-content-display system 106 modifies the campaign forfuture persons in the physical space based on determining theeffectiveness of the initial campaign presented to the person.

FIG. 10 provides additional detail regarding various components andcapabilities of the campaign management system 104 and thepersonalized-visual-content-display system 106. Generally, FIG. 10illustrates the personalized-visual-content-display system 106implemented by the campaign management system 104. In particular, asillustrated in FIG. 10, the personalized-visual-content-display system106 includes, but is not limited to a physical-item-identificationmanager 1002, an item-feature-extraction manager 1004, adigital-content-display device manager 1006, a path projection manager1008, a visual-content-item manager 1012, a content-feature-extractionmanager 1014, and a storage manager 1016.

As mentioned, the personalized-visual-content-display system 106includes the physical-item-identification manager 1002. Generally, thephysical-item-identification manager 1102 identifies physical itemswithin a set of physical items. In particular, thephysical-item-identification manager 1002 communicates with sensorslocated within the physical space to retrieve sensor data. In oneexample, the physical-item-identification manager 1002 accesses datafrom RFID sensors located within the physical space. Thephysical-item-identification manager 1002 identifies co-located physicalitems that travel together through the physical space. Thepersonalized-visual-content-display system 106 further determines theidentities of the physical items by determining identifiers from signalsassociated with the physical items.

As illustrated in FIG. 10, in some embodiments, thepersonalized-visual-content-display system 106 also includes theitem-feature-extraction manager 1004. More specifically, theitem-feature-extraction manager 1004 identifies a set of descriptionscorresponding to the set of physical items based on signals from the setof physical items. The item-feature-extraction manager 1004 utilizes alanguage-embedding model to extract a set of item features from the setof descriptions.

As further illustrated in FIG. 10, thepersonalized-visual-content-display system 106 includes thedigital-content-display device manager 1006. Generally, thedigital-content-display device manager 1006 manages a network ofdigital-content-display devices within a physical location. In oneembodiment, the digital-content-display device manager 1006 identifies adigital-content-display device in proximity to a projected location ofthe person. Furthermore, the digital-content-display device manager 1006provides, to a selected digital-content-display device, visual contentitems to display at a projected time.

As further shown in FIG. 10, the personalized-visual-content-displaysystem 106 also includes the path projection manager 1008. The pathprojection manager 1008 analyzes determined locations associated with aperson or set of physical items and/or a set of physical items togenerate a projected path of the person within the physical space.

In some embodiments, the path projection manager 1008 also includes alocation identifier 1010. The location identifier 1010 identifies aspecific projected location of the person based on the projected path.Furthermore, the location identifier 1010 also determines a projectedtime at which the person will arrive at the projected location. In someembodiments, the location identifier 1010 analyzes locationscorresponding to the set of physical items and travel times of the setof physical items between locations to predict the projected locationand the projected time.

As further shown in FIG. 10, the personalized-visual-content-displaysystem 106 illustrated in FIG. 10 also includes the visual-content-itemmanager 1012. The visual-content-item manager 1012 generally analyzes,manages, and sends visual content items to digital-content-displaydevices located within the physical space. More specifically, thevisual-content-item manager 1012 communicates with the storage manager1016 to access a collection of visual content items (i.e., visualcontent items 1020). The visual-content-item manager 1012 also managesmetadata associated with visual content items. In further embodiments,the visual-content-item manager selects the visual content item(s) todisplay via a selected digital-content-display device.

Furthermore, in one or more embodiments, thepersonalized-visual-content-display system 106 includes thecontent-feature-extraction manager 1014. Generally, thecontent-feature-extraction manager 1014 extracts a set of contentfeatures from visual content items. In particular, thecontent-feature-extraction manager 1014 accesses and utilizes animage-embedding model to generate a content embedding for each visualcontent item within a collection of visual content items. In one or moreembodiments, the content-feature-extraction manager 1014 utilizes theimage-embedding model to analyze metadata associated with visual contentitems.

In some embodiments, the storage manager 1016 includes physical itemdescriptions 1018 and visual content items 1020. In some embodiments,the physical item descriptions 1018 comprises a repository of physicalitem descriptions mapped to physical item identifiers. Morespecifically, the physical item descriptions 1018 include descriptionsfor physical items within a physical space. The visual content items1020 comprise a collection of visual content items that may potentiallybe selected for display via digital-content-display devices within aphysical space. Furthermore, the visual contentment items 1020 comprisesand associated metadata.

FIGS. 1-10, the corresponding text, and the examples provide a number ofdifferent methods, systems, devices, and non-transitorycomputer-readable media of the personalized-visual-content-displaysystem 106. In addition to the foregoing, one or more embodiments canalso be described in terms of flowcharts comprising acts foraccomplishing the particular result, as shown in FIG. 11. The actsillustrated in FIG. 11 may be performed with more or fewer acts.Further, the illustrated acts may be performed in different orders.Additionally, the acts herein may be repeated or performed in parallelwith one another or in parallel with different instances of the same orsimilar acts.

FIG. 11 illustrates a flowchart of a series of acts 1100 for retrievinga visual content item. In particular, the series of acts 1100 includesan act 1102 of determining a set of physical items selected by a person,an act 1104 of identifying a description of one or more physical items,an act 1106 of determining a projected location of the person, an act1108 of identifying a digital-content-display device in proximity to theprojected location, and an act 1110 of retrieving a visual content item.

As illustrated in FIG. 11, the series of acts 1100 includes the act 1102of determining a set of physical items selected by a person. Inparticular, the act 1102 comprises determining a set of physical itemsselected by a person within a physical space based on signals from theset of physical items. In one or more embodiments, the act 1102 furthercomprises determining the set of physical items by: receiving, from asensor of a container associated with the person, indications ofelectromagnetic signals received by the sensor from the set of physicalitems located in the container; and determining identifiers for the setof physical items prior to purchase based on the indications ofelectromagnetic signals.

In some embodiments, the act 1102 further comprises an act ofdetermining a physical item of the set of physical items by: receiving,from a sensor associated with an item-holding structure, an indicationof a signal indicating that the physical item has been removed from theitem-holding structure; and determining an identifier of the physicalitem prior to purchase based on the indication of the signal.

Additionally, in some embodiments, the act 1102 further comprises an actof determining a physical item of the set of physical items by:receiving, from a radio-frequency identification (RFID) reader, signalmeasurements reflected by a tag affixed to the physical item, whereinthe signal measurements comprise signal power, phase angle, andfrequency shift; determining one or more motion events corresponding tothe physical item based on the signal measurements; and identifying thephysical item as an item of interest by the person based on the one ormore motion events. In one or more embodiments, the act 1102 furthercomprises an act of determining the projected time by determining theprojected location at which the person will arrive within a time periodof a threshold duration.

In some embodiments, the act 1102 further comprises an act ofdetermining the set of physical items by: receiving, from one or moreelectromagnetic sensors at various locations within the physical space,indications of electromagnetic signals emitted by electromagnetic tagsassociated with the set of physical items; and determining identifiersfor the set of physical items prior to purchase based on the indicationsof electromagnetic signals.

The series of acts 1100 illustrated in FIG. 11 further comprises the act1104 of identifying a description of one or more physical items. Inparticular, the act 1104 comprises identifying a description of one ormore physical items from the set of physical items selected by theperson.

In some embodiments, the act 1104 comprises identifying a set ofdescriptions for a set of physical items selected by a person within aphysical space based on electromagnetic signals from tags associatedwith the set of physical items. In one or more embodiments, the act 1104further comprises extracting a set of item features from the set ofdescriptions.

As further illustrated in FIG. 11, the series of acts 1100 includes theact 1106 of determining a projected location of the person. Inparticular, the act 1106 comprises determining a projected location ofthe person within the physical space at a projected time. In one or moreembodiments, the act 1106 further comprises the act of determining theprojected location of the person within the physical space at theprojected time by: generating a projected path of the person within thephysical space based on detected locations at which the personinteracted with or otherwise selected a physical item from the set ofphysical items; and determining the projected location at which theperson will arrive at the projected time based on the projected path. Inone or more embodiments, the act 1106 further comprises an act ofdetermining the projected time by determining a travel time to theprojected location based on the locations corresponding to theelectromagnetic signals and travel times between the locations. Infurther embodiments, the act 1106 further comprises an act ofdetermining the projected time by determining the projected location atwhich the person will arrive within a time period of a thresholdduration.

In one or more embodiments, the act 1106 further comprises the act ofdetermining the projected location of the person within the physicalspace by: receiving, from one or more radio-frequency identification(RFID) sensors within the physical space, locations corresponding to theset of physical items over a period of time; accessing historicallocation data comprising historical locations associated with historicalpersons within the physical space; analyzing the historical locationdata to determine historical paths comprising historical locationscorresponding to the locations corresponding to the set of physicalitems; and determining the projected location based on the historicalpaths.

The series of acts 1100 further comprises the act 1108 of identifying adigital-content-display device in proximity to the projected location.In particular, the act 1108 comprises identifying adigital-content-display device in proximity to the projected location ofthe person within the physical space.

In further embodiments, the act 1108 further comprises identify adigital-content-display device in proximity to a projected location ofthe person within the physical space by: generating a projected path ofthe person within the physical space based on locations corresponding tothe electromagnetic signals from the tags associated with the set ofphysical items; and determining the projected location at which theperson will arrive at a projected time based on the projected path.Furthermore, in one or more embodiments, the act 1108 comprises an actof generating the projected path by: identifying the locationscorresponding to the electromagnetic signals by tracking detectedlocations of one or more physical items of the set of physical itemsindicated by one or more sensors detecting the electromagnetic signals;predicting one or more additional physical items the person is likely toselect based on the set of physical items; and generating the projectedpath based on the detected locations of the one or more physical itemsand one or more locations for the one or more additional physical items.

The series of acts 1100 illustrated in FIG. 11 also includes the act1110 of retrieving a visual content item. In particular, the act 1110comprises retrieving, for display via the digital-content-displaydevice, a visual content item to be viewed by the person at theprojected location and at the projected time based on the visual contentitem corresponding to the description of the one or more physical itemsfrom the set of physical items. In one or more embodiments, the act 1110further comprises the act of retrieving the visual content item by:identifying, within a co-selected-nodal graph, one or more nodesrepresenting the one or more physical items based on the description ofthe one or more physical items; determining, from the one or more nodeswithin the co-selected-nodal graph, probabilities that other physicalitems are selected together with the one or more physical items; basedon the probabilities determined from the co-selected-nodal graph,determining a commonly selected physical item; and determining thevisual content item displays the commonly selected physical item.

In one or more embodiments, the act 1110 further comprises retrieving,for display via the digital-content-display device from the collectionof visual content items, a visual content item to be viewed by theperson at the projected location and at the projected time by mapping aset of content features for the visual content item to the set of itemfeatures.

In some embodiments, the series of acts 1100 further comprisesadditional acts of identifying the description of the one or morephysical items by identifying a set of descriptions corresponding to theset of physical items based on the signals from the set of physicalitems; extracting, utilizing a language-embedding model, a set of itemfeatures from the set of descriptions; and retrieving the visual contentitem based on a set of content features for the visual content itemcorresponding to the set of item features.

In further embodiments, the series of acts 1100 comprises additionalacts of determining one or more characteristics of the person ascomprising at least one of an age, gender, spend level, size, orshopping purpose based on the set of physical items; and retrieving thevisual content item based on the one or more characteristics of theperson corresponding to the visual content item.

In one or more embodiments, the series of acts 1100 comprises additionalacts of extracting the set of item features from the set of descriptionsby generating, utilizing a language-embedding model, item embeddings forthe set of physical items from the set of descriptions; generating,utilizing an image-embedding model, a content embedding for the visualcontent item from metadata associated with the visual content item; andmapping the set of content features for the visual content items to theset of item features by determining measures of similarity among thecontent embedding and the item embeddings based on embedding distanceswithin a common embedding space among the content embedding and the itemembeddings.

In further embodiments, the series of acts 1100 comprises additionalacts of determining that the person moved a first physical item to anarea within the physical space based on a first electromagnetic signalfrom the first physical item; determine that the person did not move asecond physical item to the area within the physical space based on asecond electromagnetic signal from the second physical item; anddetermining the first physical item is part of the set of physical itemsand the second physical item is not part of the set of physical itemsbased on the person moving the first physical item to the area and notmoving the second physical item to the area.

Additionally, in one or more embodiments, the series of acts 1100further comprises the acts of determining that the person manipulated anadditional physical item based on an electromagnetic signal from theadditional physical item; extracting a subset of item features from adescription of the additional physical item; and retrieving the visualcontent item based further on the subset of item features by mapping thesubset of item features and the set of item features to the set ofcontent features.

The acts and algorithms shown in acts 206, 208, and 210 in FIG. 2 and/orthe acts 606 and 608 of FIG. 6 comprise supporting structure forperforming a step for displaying a visual content item at adigital-content-display device near a projected location of the personwithin the physical space based on the signals and the item features.

Embodiments of the present disclosure may comprise or utilize a specialpurpose or general-purpose computer including computer hardware, suchas, for example, one or more processors and system memory, as discussedin greater detail below. Embodiments within the scope of the presentdisclosure also include physical and other computer-readable media forcarrying or storing computer-executable instructions and/or datastructures. In particular, one or more of the processes described hereinmay be implemented at least in part as instructions embodied in anon-transitory computer-readable medium and executable by one or morecomputing devices (e.g., any of the media content access devicesdescribed herein). In general, a processor (e.g., a microprocessor)receives instructions, from a non-transitory computer-readable medium,(e.g., a memory), and executes those instructions, thereby performingone or more processes, including one or more of the processes describedherein.

Computer-readable media can be any available media that can be accessedby a general purpose or special purpose computer system.Computer-readable media that store computer-executable instructions arenon-transitory computer-readable storage media (devices).Computer-readable media that carry computer-executable instructions aretransmission media. Thus, by way of example, and not limitation,embodiments of the disclosure can comprise at least two distinctlydifferent kinds of computer-readable media: non-transitorycomputer-readable storage media (devices) and transmission media.

Non-transitory computer-readable storage media (devices) includes RAM,ROM, EEPROM, CD-ROM, solid state drives (“SSDs”) (e.g., based on RAM),Flash memory, phase-change memory (“PCM”), other types of memory, otheroptical disk storage, magnetic disk storage or other magnetic storagedevices, or any other medium which can be used to store desired programcode means in the form of computer-executable instructions or datastructures and which can be accessed by a general purpose or specialpurpose computer.

A “network” is defined as one or more data links that enable thetransport of electronic data between computer systems and/or modulesand/or other electronic devices. When information is transferred orprovided over a network or another communications connection (eitherhardwired, wireless, or a combination of hardwired or wireless) to acomputer, the computer properly views the connection as a transmissionmedium. Transmissions media can include a network and/or data linkswhich can be used to carry desired program code means in the form ofcomputer-executable instructions or data structures and which can beaccessed by a general purpose or special purpose computer. Combinationsof the above should also be included within the scope ofcomputer-readable media.

Further, upon reaching various computer system components, program codemeans in the form of computer-executable instructions or data structurescan be transferred automatically from transmission media tonon-transitory computer-readable storage media (devices) (or viceversa). For example, computer-executable instructions or data structuresreceived over a network or data link can be buffered in RAM within anetwork interface module (e.g., a “NIC”), and then eventuallytransferred to computer system RAM and/or to less volatile computerstorage media (devices) at a computer system. Thus, it should beunderstood that non-transitory computer-readable storage media (devices)can be included in computer system components that also (or evenprimarily) utilize transmission media.

Computer-executable instructions comprise, for example, instructions anddata which, when executed by a processor, cause a general-purposecomputer, special purpose computer, or special purpose processing deviceto perform a certain function or group of functions. In someembodiments, computer-executable instructions are executed on ageneral-purpose computer to turn the general-purpose computer into aspecial purpose computer implementing elements of the disclosure. Thecomputer executable instructions may be, for example, binaries,intermediate format instructions such as assembly language, or evensource code. Although the subject matter has been described in languagespecific to structural features and/or methodological acts, it is to beunderstood that the subject matter defined in the appended claims is notnecessarily limited to the described features or acts described above.Rather, the described features and acts are disclosed as example formsof implementing the claims.

Those skilled in the art will appreciate that the disclosure may bepracticed in network computing environments with many types of computersystem configurations, including, personal computers, desktop computers,laptop computers, message processors, hand-held devices, multiprocessorsystems, microprocessor-based or programmable consumer electronics,network PCs, minicomputers, mainframe computers, mobile telephones,PDAs, tablets, pagers, routers, switches, and the like. The disclosuremay also be practiced in distributed system environments where local andremote computer systems, which are linked (either by hardwired datalinks, wireless data links, or by a combination of hardwired andwireless data links) through a network, both perform tasks. In adistributed system environment, program modules may be located in bothlocal and remote memory storage devices.

Embodiments of the present disclosure can also be implemented in cloudcomputing environments. In this description, “cloud computing” isdefined as a model for enabling on-demand network access to a sharedpool of configurable computing resources. For example, cloud computingcan be employed in the marketplace to offer ubiquitous and convenienton-demand access to the shared pool of configurable computing resources.The shared pool of configurable computing resources can be rapidlyprovisioned via virtualization and released with low management effortor service provider interaction, and then scaled accordingly.

A cloud-computing model can be composed of various characteristics suchas, for example, on-demand self-service, broad network access, resourcepooling, rapid elasticity, measured service, and so forth. Acloud-computing model can also expose various service models, such as,for example, Software as a Service (“SaaS”), Platform as a Service(“PaaS”), and Infrastructure as a Service (“IaaS”). A cloud-computingmodel can also be deployed using different deployment models such asprivate cloud, community cloud, public cloud, hybrid cloud, and soforth. In this description and in the claims, a “cloud-computingenvironment” is an environment in which cloud computing is employed.

FIG. 12 illustrates a block diagram of a computing device 1200 that maybe configured to perform one or more of the processes described above.One will appreciate that one or more computing devices such as thecomputing device 1200 may implement thepersonalized-visual-content-display system 106 and the campaignmanagement system 104. As shown by FIG. 12, the computing device 1200can comprise a processor 1202, a memory 1204, a storage device 1206, anI/O interface 1208, and a communication interface 1210, which may becommunicatively coupled by way of a communication infrastructure 1212.In certain embodiments, the computing device 1200 can include fewer ormore components than those shown in FIG. 12. Components of the computingdevice 1200 shown in FIG. 12 will now be described in additional detail.

In one or more embodiments, the processor 1202 includes hardware forexecuting instructions, such as those making up a computer program. Asan example, and not by way of limitation, to execute instructions fordynamically modifying workflows, the processor 1202 may retrieve (orfetch) the instructions from an internal register, an internal cache,the memory 1204, or the storage device 1206 and decode and execute them.The memory 1204 may be a volatile or non-volatile memory used forstoring data, metadata, and programs for execution by the processor(s).The storage device 1206 includes storage, such as a hard disk, flashdisk drive, or other digital storage device, for storing data orinstructions for performing the methods described herein.

The I/O interface 1208 allows a user to provide input to, receive outputfrom, and otherwise transfer data to and receive data from computingdevice 1200. The I/O interface 1208 may include a mouse, a keypad or akeyboard, a touch screen, a camera, an optical scanner, networkinterface, modem, other known I/O devices or a combination of such I/Ointerfaces. The I/O interface 1208 may include one or more devices forpresenting output to a user, including, but not limited to, a graphicsengine, a display (e.g., a display screen), one or more output drivers(e.g., display drivers), one or more audio speakers, and one or moreaudio drivers. In certain embodiments, the I/O interface 1208 isconfigured to provide graphical data to a display for presentation to auser. The graphical data may be representative of one or more graphicaluser interfaces and/or any other graphical content as may serve aparticular implementation.

The communication interface 1210 can include hardware, software, orboth. In any event, the communication interface 1210 can provide one ormore interfaces for communication (such as, for example, packet-basedcommunication) between the computing device 1200 and one or more othercomputing devices or networks. As an example, and not by way oflimitation, the communication interface 1210 may include a networkinterface controller (NIC) or network adapter for communicating with anEthernet or other wire-based network or a wireless NIC (WNIC) orwireless adapter for communicating with a wireless network, such as aWI-FI.

Additionally, the communication interface 1210 may facilitatecommunications with various types of wired or wireless networks. Thecommunication interface 1210 may also facilitate communications usingvarious communication protocols. The communication infrastructure 1212may also include hardware, software, or both that couples components ofthe computing device 1200 to each other. For example, the communicationinterface 1210 may use one or more networks and/or protocols to enable aplurality of computing devices connected by a particular infrastructureto communicate with each other to perform one or more aspects of theprocesses described herein. To illustrate, the digital content campaignmanagement process can allow a plurality of devices (e.g., a clientdevice and server devices) to exchange information using variouscommunication networks and protocols for sharing information such asdigital messages, user interaction information, engagement metrics, orcampaign management resources.

In the foregoing specification, the present disclosure has beendescribed with reference to specific exemplary embodiments thereof.Various embodiments and aspects of the present disclosure(s) aredescribed with reference to details discussed herein, and theaccompanying drawings illustrate the various embodiments. Thedescription above and drawings are illustrative of the disclosure andare not to be construed as limiting the disclosure. Numerous specificdetails are described to provide a thorough understanding of variousembodiments of the present disclosure.

The present disclosure may be embodied in other specific forms withoutdeparting from its spirit or essential characteristics. The describedembodiments are to be considered in all respects only as illustrativeand not restrictive. For example, the methods described herein may beperformed with less or more steps/acts or the steps/acts may beperformed in differing orders. Additionally, the steps/acts describedherein may be repeated or performed in parallel with one another or inparallel with different instances of the same or similar steps/acts. Thescope of the present application is, therefore, indicated by theappended claims rather than by the foregoing description. All changesthat come within the meaning and range of equivalency of the claims areto be embraced within their scope.

1. A non-transitory computer readable medium for providing personalizedvisual content items for display on digital-content-display devices, thenon-transitory computer readable medium comprising instructions that,when executed by at least one processor, cause a computing device to:determine a set of physical items selected by a person within a physicalspace based on signals from the set of physical items; identify adescription of one or more physical items from the set of physical itemsselected by the person; determine a projected location of the personwithin the physical space at a projected time; identify adigital-content-display device in proximity to the projected location ofthe person within the physical space; and retrieve, for display via thedigital-content-display device, a visual content item to be viewed bythe person at the projected location and at the projected time based onthe visual content item corresponding to the description of the one ormore physical items from the set of physical items.
 2. Thenon-transitory computer readable medium as recited in claim 1, furthercomprising instructions that, when executed by the at least oneprocessor, cause the computing device to determine the set of physicalitems by: receiving, from a sensor of a container associated with theperson, indications of electromagnetic signals received by the sensorfrom the set of physical items located in the container; and determiningidentifiers for the set of physical items prior to purchase based on theindications of electromagnetic signals.
 3. The non-transitory computerreadable medium as recited in claim 1, further comprising instructionsthat, when executed by the at least one processor, cause the computingdevice to determine a physical item of the set of physical items by:receiving, from a sensor associated with an item-holding structure, anindication of a signal indicating that the physical item has beenremoved from the item-holding structure; and determining an identifierof the physical item prior to purchase based on the indication of thesignal.
 4. The non-transitory computer readable medium as recited inclaim 1, further comprising instructions that, when executed by the atleast one processor, cause the computing device to determine a physicalitem of the set of physical items by: receiving, from a radio-frequencyidentification (RFID) reader, signal measurements reflected by a tagaffixed to the physical item, wherein the signal measurements comprisesignal power, phase angle, and frequency shift; determining one or moremotion events corresponding to the physical item based on the signalmeasurements; and identifying the physical item as an item of interestby the person based on the one or more motion events.
 5. Thenon-transitory computer readable medium as recited in claim 1, furthercomprising instructions that, when executed by the at least oneprocessor, cause the computing device to: identify the description ofthe one or more physical items by identifying a set of descriptionscorresponding to the set of physical items based on the signals from theset of physical items; extract, utilizing a language-embedding model, aset of item features from the set of descriptions; and retrieve thevisual content item based on a set of content features for the visualcontent item corresponding to the set of item features.
 6. Thenon-transitory computer readable medium as recited in claim 1, furthercomprising instructions that, when executed by the at least oneprocessor, cause the computing device to: determine one or morecharacteristics of the person as comprising at least one of an age,gender, spend level, size, or shopping purpose based on the set ofphysical items; and retrieve the visual content item based on the one ormore characteristics of the person corresponding to the visual contentitem.
 7. The non-transitory computer readable medium as recited in claim1, further comprising instructions that, when executed by the at leastone processor, cause the computing device to determine the projectedlocation of the person within the physical space at the projected timeby: generating a projected path of the person within the physical spacebased on detected locations at which the person interacted with orotherwise selected a physical item from the set of physical items; anddetermining the projected location at which the person will arrive atthe projected time based on the projected path.
 8. The non-transitorycomputer readable medium as recited in claim 1, further comprisinginstructions that, when executed by the at least one processor, causethe computing device to retrieve the visual content item by:identifying, within a co-selected-nodal graph, one or more nodesrepresenting the one or more physical items based on the description ofthe one or more physical items; determining, from the one or more nodeswithin the co-selected-nodal graph, probabilities that other physicalitems are selected together with the one or more physical items; basedon the probabilities determined from the co-selected-nodal graph,determining a commonly selected physical item; and determining thevisual content item displays the commonly selected physical item.
 9. Asystem comprising: at least one memory device comprising descriptions ofphysical items, a collection of visual content items, and correspondingsets of content features for the collection of visual content items; andat least one server device configured to cause the system to: identify aset of descriptions for a set of physical items selected by a personwithin a physical space based on electromagnetic signals from tagsassociated with the set of physical items; extract a set of itemfeatures from the set of descriptions; identify adigital-content-display device in proximity to a projected location ofthe person within the physical space by: generating a projected path ofthe person within the physical space based on locations corresponding tothe electromagnetic signals from the tags associated with the set ofphysical items; and determining the projected location at which theperson will arrive at a projected time based on the projected path; andretrieve, for display via the digital-content-display device from thecollection of visual content items, a visual content item to be viewedby the person at the projected location and at the projected time bymapping a set of content features for the visual content item to the setof item features.
 10. The system as recited in claim 9, wherein the atleast one server device is further configured to cause the system to:extract the set of item features from the set of descriptions bygenerating, utilizing a language-embedding model, item embeddings forthe set of physical items from the set of descriptions; generate,utilizing an image-embedding model, a content embedding for the visualcontent item from metadata associated with the visual content item; andmap the set of content features for the visual content items to the setof item features by determining measures of similarity among the contentembedding and the item embeddings based on embedding distances within acommon embedding space among the content embedding and the itemembeddings.
 11. The system as recited in claim 9, wherein the at leastone server device is further configured to cause the system to generatethe projected path by: identifying the locations corresponding to theelectromagnetic signals by tracking detected locations of one or morephysical items of the set of physical items indicated by one or moresensors detecting the electromagnetic signals; predicting one or moreadditional physical items the person is likely to select based on theset of physical items; and generating the projected path based on thedetected locations of the one or more physical items and one or morelocations for the one or more additional physical items.
 12. The systemas recited in claim 9, wherein the at least one server device is furtherconfigured to cause the system to determine the projected time bydetermining a travel time to the projected location based on thelocations corresponding to the electromagnetic signals and travel timesbetween the locations.
 13. The system as recited in claim 9, wherein theat least one server device is further configured to cause the system todetermine the projected time by determining the projected location atwhich the person will arrive within a time period of a thresholdduration.
 14. The system as recited in claim 9, wherein the at least oneserver device is further configured to cause the system to determine theset of physical items by: receiving, from one or more electromagneticsensors at various locations within the physical space, indications ofelectromagnetic signals emitted by electromagnetic tags associated withthe set of physical items; and determining identifiers for the set ofphysical items prior to purchase based on the indications ofelectromagnetic signals.
 15. The system as recited in claim 9, whereinthe at least one server device is further configured to cause the systemto determine the projected location of the person within the physicalspace by: receiving, from one or more radio-frequency identification(RFID) sensors within the physical space, locations corresponding to theset of physical items over a period of time; accessing historicallocation data comprising historical locations associated with historicalpersons within the physical space; analyzing the historical locationdata to determine historical paths comprising historical locationscorresponding to the locations corresponding to the set of physicalitems; and determining the projected location based on the historicalpaths.
 16. The system as recited in claim 9, wherein the at least oneserver device is further configured to cause the system to: determinethat the person moved a first physical item to an area within thephysical space based on a first electromagnetic signal from the firstphysical item; determine that the person did not move a second physicalitem to the area within the physical space based on a secondelectromagnetic signal from the second physical item; and determine thefirst physical item is part of the set of physical items and the secondphysical item is not part of the set of physical items based on theperson moving the first physical item to the area and not moving thesecond physical item to the area.
 17. The system as recited in claim 9,wherein the at least one server device is further configured to causethe system to: determine that the person manipulated an additionalphysical item based on an electromagnetic signal from the additionalphysical item; extract a subset of item features from a description ofthe additional physical item; and retrieve the visual content item basedfurther on the subset of item features by mapping the subset of itemfeatures and the set of item features to the set of content features.18. In a digital medium environment for providing personalized visualcontent items for display on digital-content-display devices, acomputer-implemented method comprising: receiving signals from tagsassociated with physical items selected by a person within a physicalspace; extracting item features from a description of one or morephysical items from the physical items selected by the person; andperforming a step for displaying a visual content item at adigital-content-display device near a projected location of the personwithin the physical space based on the signals and the item features.19. The computer-implemented method as recited in claim 18, whereinreceiving the signals from the tags associated with the physical itemscomprises receiving, from a sensor of a container associated with theperson, indications of radio-frequency identification (RFID) signalsreceived by the sensor from the physical items located in the container.20. The computer-implemented method as recited in claim 18, whereinreceiving the signals from the tags associated with the physical itemscomprises receiving the signals from attached tags affixed to thephysical items that have been picked up and set down by the person.